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FOREWARD 


As  the  fidelity  increases  for  tactical  aircraft  training  simulation  devices,  especially 
visual  simulation,  the  contribution  of  Force  Cueing  needs  to  be  established.  The  pilot  in 
a  fighter  aircraft  is  constantly  experiences  various  forces  on  the  body.  These  forces 
provide  cues,  consciously  and  unconsciously,  about  the  accelerations  of  the  aircraft 
resulting  from  pilot  control  input,  the  environment  and  aircraft  state  including  failures.  In 
a  static  simulator,  these  cues  are  not  present. 

In  a  simulator  with  force  cueing  devices,  such  as  motion  platform,  dynamic  seat, 
G-suit,  etc.,  the  contribution  of  force  cues  to  pilot  performance  and  training  is  virtually 
unknown.  The  issue  of  Force  Cueing  contribution  for  tactical  aircraft  training  devices  has 
been  unanswered  for  twenty-five  years.  As  the  use  of  training  devices  increases  and  as 
these  devices  are  being  used  for  combat  training  and  rehearsal,  the  contributions  of  force 
cueing  need  to  be  established  to  ensure  that  the  combat  pilot  behavior  and  performance  in 
the  simulator  will  be  as  close  as  possible  to  what  as  experienced  in  the  aircraft. 

As  a  result  of  an  initial  force  cueing  study  conducted  for  the  US  Air  Force  by 
SIMTEC,  Inc.,  Manassas,  Virginia,  it  was  concluded  that  the  effectiveness  of  potential 
force  cueing  devices  could  only  be  determined  by  evaluation  in  a  mission  context  with 
experienced  operational  pilot.  The  Air  Force  had  already  successfully  performed  similar 
operational  evaluations  of  visual  combat  simulators,  under  a  program  referred  to  as  “Vis- 
Eval”  using  these  same  ground  rules.  Based  upon  the  success  of  these  efforts,  it  appeared 
logical  that  a  force  evaluation  (Force-Eval)  process  might  be  useful  for  evaluation  of 
force  cueing  devices.  However,  it  was  recognized  that  force  cueing  effects  are  much 
more  subtle  and  harder  to  isolate  than  visual  cues  and,  therefore,  would  be  more  difficult 
to  evaluate.  Because  of  this  difficulty  and  other  factors,  such  as  the  pilots  may  be 
unaware  of  changes  in  their  control  strategy  as  a  function  of  force  cueing,  it  was 
determined  to  be  essential  that  a  force  cueing  evaluation  be  based  largely  on  objective 
data  collection. 

This  purpose  of  the  current  study  effort  was  to  develop  a  method  for  evaluating 
the  effectiveness  of  various  force  cueing  devices  in  a  flight  simulator.  The  concept 
developed  included  measurement  of  pilot  behavior,  performance,  physiology  and 
subjective  pilot  opinion  to  evaluate  system  effectiveness. 

A  Trial  Force-Eval  was  conducted  in  a  fighter  simulator  to  validate  the  method 
including  identifying  which  pilot  behaviors  could  best  be  measured  and  how  the  data 
could  be  collected  and  analyzed.  Five  task  scenarios  were  flown  by  experienced  fighter 
pilots  with  and  without  the  presence  of  force  cueing.  Control  activity  and  vehicle  state 
data  were  analyzed  to  examine  the  effects  of  force  cueing  on  pilot  performance  and 
control  behavior.  A  debriefing  questionnaire  was  used  to  elicit  the  pilot’s  subjective 
evaluation.  The  results  of  the  Trial  Force-Eval  indicated  that  the  presence  of  force  cueing 
improved  pilot  performance,  control  behavior,  and  made  the  simulator  more  operationally 
realistic. 
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SECTION  1  INTRODUCTION 


The  use  of  ground  based  training  devices  for  fighter  aircraft  in  the  United  States 
Air  Force  (USAF)  has  principally  been  limited  to  emergency  procedures,  part  task,  and 
non-visual  mission  training.  For  that  reason,  the  need  for  force  cueing  has  also  been 
limited.  Attempts  in  the  past  to  use  force  cueing  devices,  such  as  platform  motion  and 
dynamic  seats,  have  met  with  limited  success  for  a  variety  of  reasons  including  poor 
hardware  performance,  ineffective  drive  algorithms  and  excessive  transport  delays. 

As  the  USAF  transitions  into  tactical  combat  mission  training  using  multiple 
networked  simulators,  the  requirement  for  force  cueing  needs  to  be  assessed.  The 
contribution  of  force  cueing  in  training  simulators  is  not  well  understood.  The 
Commander  of  Air  Combat  Command,  during  a  Distributed  Mission  Training  conference 
in  July  1997,  expressed  concern  that  without  force  cueing,  pilots  would  pull  nine  or  ten 
g’s  in  a  simulator  without  being  aware  that  they  were  doing  so. 

The  objective  of  a  training  simulator  is  to  allow  the  pilot  to  develop,  experience 
and  reinforce  the  techniques  and  habit  patterns  that  will  be  needed  for  correct 
performance  in  the  aircraft.  Generally,  for  any  task  in  the  aircraft,  there  are  unique  sets  of 
critical  cues  to  which  a  pilot  responds  in  order  to  accomplish  the  task.  Pilots  are  very 
adaptable.  If  the  proper  cues  are  not  provided  in  a  simulator  that  would  be  normally  use 
in  the  aircraft,  the  pilot  will  find  other  cues  in  order  to  accomplish  the  task.  This 
generally  does  not  result  in  optimal  performance  and  may  lead  to  different  control 
strategies  than  used  in  the  aircraft.  Obviously,  this  is  not  optimal  for  training  since  the 
pilot  is  simply  learning  to  fly  the  simulator  rather  than  the  aircraft  and  may  have  to  later 
relearn  in  the  aircraft. 

The  evaluation  of  force  cueing  is  difficult.  Pilots  largely  do  not  understand  which 
force  cues  they  use  or  how  they  use  them  while  flying  the  aircraft.  In  the  simulator,  the 
pilots  cannot  always  articulate  the  impact  of  incorrect  or  missing  force  cues  on  their 
performance  or  control  strategy.  Thus,  an  analytical  evaluation  method  is  needed  to 
determine  the  contributions  of  various  force  cueing  devices  in  a  tactical  training 
simulator. 

The  purpose  of  this  program  was  to  develop  a  method  for  conducting  force  cueing 
evaluations.  The  program  included  a  Trial  Force-Eval  that  was  conducted  to  validate  that 
evaluation  method.  This  method  includes  both  procedures  and  criteria  for  evaluating  the 
effectiveness  of  various  force  cueing  devices,  such  as  platform  motion  systems,  dynamic 
seats,  etc.,  for  tactical  mission  simulation.  The  Trial  Force-Eval  included  a  dynamic  seat 
referred  to  as  the  ALCOGS,  actuation  of  the  pilot’s  anti-g  suit  and  Combat  Edge. 

Combat  Edge  is  a  positive  pressure  breathing  system  used  on  current  fighter  aircraft. 
Supporting  the  Trial  Force-Eval  was  a  simplified  fighter  cockpit  with  F-15C  dynamics,  a 
limited  field-of-view  rear  projection  screen  display  and  an  Evans  and  Sutherland  4530 
Image  Generator.  The  Trial  Force-Eval  was  conducted  at  the  Control  Integration  and 
Assessment  Branch  Simulation  Facility,  Air  Vehicles  Directorate,  Air  Force  Research 
Laboratory,  Wright  Patterson  AFB,  Ohio. 


l 


After  the  completion  of  this  Trial  Force-Eval,  a  complete  evaluation  needs  to  be 
accomplished  on  a  simulator  which  includes  a  full  spectrum  of  force  cueing  devices  such 
as  performance  platform  motion,  a  dynamic  seat,  and  a  full  field  of  regard  visual  system. 
This  Force-Eval  should  determine  the  contributions  of  the  various  force  cueing 
technologies,  individually  and  in  combinations,  in  a  simulated  combat  mission  training 
environment. 

This  report  will  describe  an  overall  approach  to  the  development  of  an  evaluation 
method  and  a  Trial  Force-Eval,  which  examined  a  representative  subset  of  the  overall 
evaluation  method. 
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SECTION  2  APPROACH 


Ideally,  the  determination  of  the  need  for  training  device  capability  would  be 
made  based  on  transfer  of  training  studies.  However,  evaluations  that  isolate  the  effects 
of  different  capabilities  of  a  simulation  device  on  training  are  difficult  to  develop  and 
implement. 

An  alternative  to  transfer  of  training  studies  is  to  evaluate  force  cueing 
effectiveness  subjectively,  similar  to  the  USAF  Vis-Eval  program.  Vis-Eval  uses  highly 
experienced  instructor  pilots  to  fly  tactical  missions  in  candidate  visual  simulators.  These 
pilots  rate  the  potential  of  the  visual  system  to  support  student  training  of  each  task 
within  representative  combat  missions  (Brown,  1994).  In  applying  this  technique,  it  must 
be  recognized  that  the  effects  of  force  cueing  are  often  subtle  and,  in  many  cases,  may  not 
be  outwardly  recognized  by  the  pilot.  Another  method  to  evaluate  force  cueing 
effectiveness  is  to  measure  differences  in  pilot  performance  with  and  without  different 
force  cueing  inputs.  However,  this  technique  by  itself  may  not  be  sufficient  since  pilots 
may  compensate  for  the  lack  of  certain  cues  by  altering  their  control  strategy.  A  pilot 
may  not  be  aware  that  he  or  she  is  compensating  for  the  lack  of  forces  that  would 
normally  be  present  in  the  aircraft.  Outwardly,  performance  without  force  cueing  may  be 
similar  to  what  takes  place  in  the  aircraft  (or  a  simulator  with  adequate  cueing);  however, 
pilot  behavior  in  the  form  of  control  strategy  may  be  far  different  than  the  aircraft.  In 
essence,  the  pilot  may  be  learning  to  fly  the  simulator  and  later  have  to  relearn  in  the 
aircraft. 

What  appears  to  be  an  optimum  approach  to  evaluating  force  cueing  devices  is  to 
measure  a  combination  of  changes  in  pilot  performance,  physiology,  behavior  and 
subjective  opinion  while  flying  a  simulator  with  and  without  different  force  cue  inputs 
during  typical  tactical  missions.  It  is  possible  to  analyze  how  the  lack  of  force  cues  and 
the  presence  of  different  simulated  force  cues  affected  pilot  behavior  and  performance  as 
compared  to  full  body  force  cueing  in  the  aircraft.  This  can  be  accomplished  by 
comparing  data  taken  in  the  simulator  with  data  taken  in  the  aircraft  while  performing 
similar  missions  and  maneuvers. 

The  Trial  Force-Eval  did  not  have  the  luxury  of  having  available  flight  testing  to 
collect  data  which  then  could  be  used  as  a  standard  to  compare  with  simulator  data. 
Therefore,  the  effects  of  force  cueing  could  only  be  compared  by  the  presence  or  lack 
thereof  in  the  simulator. 

The  approach  taken  to  develop  and  validate  an  evaluation  method  to  determine 
effectiveness  of  force  cueing  devices  was  conducted  using  several  steps.  The  first  step 
was  to  identify  potential  cueing  devices  that  could  become  part  of  the  Trial  Force-Eval 
within  the  limits  of  the  program.  Four  such  devices  were  identified  including  (1)  a 
dynamic  seat  referred  to  as  the  ALCOGS,  (2)  the  Combat  Edge  positive  pressure 
breathing  system,  (3)  anti-g  suit  and  (4)  Clark  Audio  transducer.  A  detailed  description 
of  these  devices  is  provided  in  Section  5,  “Trial  Force-Eval  System  and  Evaluation”. 
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The  Clark  Audio  transducer  was  not  included  in  the  Trial  Force-Eval  because  of 
limited  quality  of  the  source  data  representing  the  aircraft’s  audio  range.  This  was  a 
disappointment  since  the  transducer  appeared  to  have  great  potential  as  a  low  cost  means 
of  providing  force  cue  inputs  such  as  cobble  stone  effect  during  ground  operation, 
weapon  firing  effects,  system  actuation  and  higher  frequency  information  such  as  engine 
operation. 

Concurrently  with  the  effort  to  select  the  devices  that  would  form  the  basis  for  the 
Trial  Force-Eval,  an  analysis  was  conducted  to  identify  and  tabulate  the  different  forces 
encountered  in  a  fighter  aircraft.  The  results  are  documented  in  Section  3,  “Types  and 
Effects  of  Force  Cues  Associated  with  Fighter  Aircraft”. 

Since  the  purpose  of  the  effort  was  to  develop  a  method  for  evaluating  force 
cueing  effectiveness,  it  was  necessary  that  criteria  be  established  to  decide  what  was  and 
was  not  effectiveness.  Section  4,  entitled  “Criteria  for  Determining  Force  Cueing 
Effectiveness”,  discusses  the  various  criteria  which  were  explored  and  later  validated. 

Once  the  cueing  devices  were  identified  for  the  Trial  Force-Eval,  steps  were  taken 
to  integrate  them  into  a  fighter  simulation  system.  This  system  consisted  of  F-15  aero¬ 
dynamics,  a  generic  fighter  cockpit  with  appropriate  controls  and  avionics  systems.  For  a 
description  of  the  hardware  used  for  the  Trial  Force-Eval,  refer  to  Section  5  entitled 
“Trial  Force-Eval  System  and  Evaluation”. 

Development  and  evaluation  of  drive  algorithms  were  a  critical  part  of  the  Trial 
Force-Eval.  The  drive  algorithms,  more  than  hardware  performance,  determines  the 
effectiveness  of  force  cueing  systems.  Expert  engineering  pilots  and  simulation 
specialists  worked  together  using  a  trial  and  error  procedure  to  define  and  refine  the 
algorithms.  The  most  critical  system  requiring  the  greatest  amount  of  effort  was  the 
dynamic  seat.  Algorithms  developed  earlier  for  this  seat  did  not  address  problems 
encountered  when  the  seat  was  used  in  a  full  tactical  mission  environment.  As  an 
example,  tracking  algorithms  that  appeared  to  work  well  in  a  1-g  environment  did  not 
work  at  all  well  when  high-g  pursuit  tracking  was  performed. 

From  the  beginning  of  the  program  it  was  recognized,  that  not  all  flight  tasks  or 
portions  of  the  aircraft  flight  envelope  would  be  sensitive  to  force  cueing.  Therefore, 
tasks  scenario  had  to  be  chosen  where  forces  would  conceivably  have  an  effect  on  the 
pilot’s  behavior  and/or  performance.  Task  scenarios  are  discussed  in  Section  5. 
Instrumentation  is  another  important  area  where  trial  and  error  was  necessary  to 
determine  what  data  would  produce  useful  information. 

The  Trial  Force-Eval  was  conducted  with  a  limited  number  of  highly  experienced 
fighter  pilots.  Performance  data,  control  behavior  and  subjective  data  were  analyzed  to 
identify  differences  between  a  mission  flown  with  and  without  force  cueing.  Descriptive 
statistics  were  computed  to  investigate  differences.  Power  spectral  analyses  were 
performed  on  stick  activity  to  identify  differences  in  control  behavior.  Subjective 
debriefing  questionnaires  were  reviewed  and  summarized. 
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SECTION  3  TYPES  AND  EFFECTS  OF  DIFFERENT  FORCE  CUES 
ASSOCIATED  WITH  FIGHTER  AIRCRAFT  FLIGHT 


Force  cues  experienced  in  a  fighter  aircraft  vary  in  frequency  from  the  low 
frequency  high-g  sustained  forces  associated  with  high  performance  maneuvering  to 
audio  frequency  range  vibrations.  The  Force-Eval  covered  the  full  range  of  force  cueing 
stimulus.  It  examined  the  application  of  different  devices  that  may  provide  the  pilot  with 
useful  cues.  In  addition  to  differences  in  frequency,  force  cues  may  also  be  broken  down 
other  ways  such  as  maneuver  cues  and  disturbance  cues.  Maneuver  forces  or  motion  may 
be  defined  as  that  motion  that  arises  within  the  control  loop  as  a  result  of  flight  control 
inputs  by  the  pilot.  Disturbance  forces  or  motion,  on  the  other  hand,  arise  outside  of  the 
control  loop  as  a  result  of  atmospheric  turbulence  or  failure  of  some  component  of  the 
aircraft  that  causes  an  unexpected  motion  of  the  aircraft.  Disturbance  cues  also  result 
from  changes  in  aircraft  state  such  as  firing  of  stores  or  actuation  of  systems  such  as  dive 
brakes. 

Force  cues  are  generally  perceived  by  three  human  sensors;  visual,  vestibular  and 
tactual.  The  pilot  senses  motion  from  the  visual  scene  information  by  vection.  Vection  is 
an  illusionary  sensation  of  self-motion  produced  by  translatory  motion  of  the  visual 
scene.  The  sensation  that  you  are  moving  when  stopped  at  a  stop  light  caused  by 
perceiving  the  car  next  to  you  moving  in  the  periphery  may  be  classified  as  a  vection  cue. 
Vection  cues  are  said  to  be  more  powerful  when  the  visual  translation  occurs  in  the 
distance  and  in  the  periphery  (Boff,  1988). 

Acceleration  inputs  are  provided  by  the  vestibular  system  (imbedded  in  the 
temporal  bones  on  each  side  of  the  head  near  the  inner  ear)  which  includes  the  semi¬ 
circular  canals  and  the  otolith  organs.  For  extremely  small  maneuver  corrections,  the 
forces  may  be  too  small  to  be  sensed  by  the  vestibular  system.  The  minimum  threshold  at 
which  acceleration  is  sensed  by  the  vestibular  system  is  approximately  0.3  degrees  per 
second  per  second  (Boff,  1988). 

Tactual  inputs  include  touch  that  is  mediated  by  receptors  in  the  skin  (tactile 
input)  and  sense  of  movement  and  position  of  the  limbs  and  other  body  parts,  arising 
from  stimulation  of  receptors  in  joints,  muscles,  and  tendons  (kinesthetic). 

In  the  aircraft,  the  pilot  has  access  to  the  input  of  all  three  sensors.  Although  they 
may  often  provide  redundant  inputs  to  the  brain,  these  inputs  tend  to  reinforce  each  other. 
The  reaction  time  of  the  vestibular  and  tactual  input  appear  to  be  faster  than  the  visual 
input  (Boff,  1988). 
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3.1  Sustained  Maneuver  High-g  Forces 

High-g  maneuvers  result  in  very  low  frequency  maneuver  cueing.  The  sustained 
high-g  maneuvering  of  a  fighter  aircraft  significantly  affect  the  pilot’s  performance  and 
physiology.  Few  of  the  force  cues  associated  with  high-g  maneuvering  can  effectively  be 
simulated  in  a  ground-based  trainer.  Some  of  the  effects  associated  with  these  forces 
include: 

(1)  Blood  Pooling  -  During  high-g  maneuvering,  the  blood  pools  in  the  lower 
extremities  which  may  lead  to  loss  of  consciousness.  The  anti-g  suit  worn  by  the 
pilot  provides  counter-pressure  forces  through  inflation  to  reduce  blood  pooling. 
During  such  maneuvering,  the  counteracting  forces  between  the  body  and  the  anti- 
g  suit  provide  the  pilot  with  cues  as  to  relatively  how  much  g  force  the  aircraft  is 
pulling.  In  a  simulator,  the  g-suit  can  be  activated  to  provide  sustained  g  force 
cues  to  the  pilot.  However,  the  amount  of  force  applied  by  the  suit  must  be  scaled 
down  since  the  forces  of  the  blood  are  not  working  against  the  g-suit  forces. 

Another  device  worn  by  the  pilot  in  some  fighter  aircraft  to  counteract  the 
effects  of  high-g  forces  is  the  Combined  Advanced  Technology  Enhanced  Design 
G  Ensemble  (Combat  Edge).  Combat  Edge,  is  shown  in  Figure  2.  It  consists  of 

(1)  a  positive  pressure  breathing  system,  (2)  a  bladder  fitted  to  the  rear  inside  of 
the  flight  helmet  which  pulls  on  a  strap  to  hold  the  oxygen  mask  in  place  during 
high-g  maneuvering  and  (3)  an  inflatable  chest  bladder  to  apply  pressure  to  the 
pilot’s  chest  during  high-g  maneuvers. 

An  anti-g  suit  together  with  the  Combat  Edge  was  activated  during  the 
Trial  Force-Eval. 

(2)  Pilot  Pressed  Into  Seat  -  During  high-g  maneuvering,  g  forces  press  the  pilot 
into  the  seat,  applying  pressure  on  the  buttocks  and  the  thighs.  In  addition,  when 
the  pilot  is  pressed  into  the  seat,  his  eyepoint  is  slightly  lowered.  The  dynamic 
seat  provides  the  pilot  with  an  indication  of  the  increased  pressure  being  applied 
to  the  buttocks  and  a  lowering  of  eye  height.  During  the  Trial  Force-Eval, 
inflating  the  seat  bladder  increased  buttock  pressure. 

(3)  Pilot  Limbs  Become  Heavy  -  In  the  aircraft,  the  pilot’s  arms  and  legs  become 
extremely  heavy  during  high-g  maneuvering  and  it  becomes  difficult  to  move  in 
the  cockpit.  Generally,  the  pilot  is  able  to  move  around,  but  must  do  so 
differently.  There  has  been  some  experimentation  in  simulators  to  apply  loading 
to  the  pilot’s  limbs  through  attachment  of  devices  such  as  cables  that  restrict  the 
pilot’s  movement.  Such  devices  have  generally  been  found  very  cumbersome. 

(4)  Increased  Head  and  Helmet  Weight  -  The  pilot’s  head  and  helmet,  like  the 
limbs,  increase  in  weight  proportional  to  the  number  of  g’s  pulled  in  the  aircraft. 
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This  increase  makes  it  difficult  for  the  pilot  to  turn  his  or  her  head.  This  also  puts 
a  strain  on  the  neck.  Efforts  have  been  made  to  artificially  force  load  the  pilot’s 
helmet  during  simulated  high-g  maneuvering  (Ashworth,  1978). 

(5)  Effects  of  High-g  Forces  on  Vision  -  During  high-g  maneuvering,  blood  flows 
to  the  lower  extremities  causing  a  loss  of  vision.  This  loss  of  vision  happens  in 
two  stages  starting  with  degradation  of  peripheral  vision  resulting  in  tunnel  vision 
and  finally  a  total  loss  of  vision  or  blackout.  Individual  pilot’s  resistance  to  loss 
of  vision  varies  as  function  of  conditioning,  training  and  recent  experience.  In 
addition,  shorter  pilots  generally  have  an  advantage  over  taller  pilots  because  of 
the  shorter  distance  from  the  brain  to  the  heart.  It  is  possible  in  a  simulator  to 
provide  the  gray  out/blackout  effect  by  dimming  the  visual  system  and,  with 
certain  visual  display  hardware  capabilities,  simulate  the  tunnel  vision  effect.  A 
problem  with  simulation  of  these  effects  is  that  it  is  difficult  to  vary  the  onset  of 
these  vision  loss  effects  since  they  are  a  function  of  individual  pilot  resistance.  In 
the  aircraft,  the  pilot  is  able  to  delay  the  effect  of  vision  loss  by  performing  the 
“M-l”  maneuver,  a  straining  and  grunting  action  to  increase  tolerance  to  high 
positive  vertical  acceleration.  A  method  for  sensing  through  myoelectric  signals 
when  the  pilot  is  performing  the  maneuver  and  varying  loss  of  a  vision  as  a 
function  of  the  straining  has  been  proposed  (Albery,  Gum,  1978). 

3.2  Disturbance  Forces 

Disturbance  force  cues  result  from  aircraft  motion  that  is  independent  of  the 
pilot’s  inputs  to  the  flight  control  system.  These  cues  take  different  forms  including 
turbulence,  buffet,  ground  effects,  or  vibration.  Disturbance  cues  may  be  either 
correlated  or  uncorrelated.  Correlated  disturbances  are  a  result  of  an  event  such  as  an 
engine  out,  dive  brake  deployment,  etc.  They  provide  the  pilot  with  an  alerting  cue  that 
an  event  has  taken  place.  Uncorrelated  disturbances  such  as  turbulence  are  not  related  to 
an  event  and  therefore  do  not  provide  the  pilot  an  alerting  cue.  Such  cues  can  cause 
fatigue  and  increase  workload. 

3.2.1  Buffet 

Buffet  results  from  airflow  across  the  aircraft  surfaces  causing  an  aerodynamic 
disturbance  that  moves  the  aircraft.  Buffet  cues  are  correlated  disturbances,  which 
provide  information  on  current  conditions  such  as  angle  of  attack  and  airspeed.  They  also 
provide  feedback  as  to  current  configuration  and  configuration  changes,  i.e.  landing  gear, 
flaps,  stores,  speed  brakes,  etc.  Buffet  can  provide  the  pilot  with  critical  alerting  cues. 
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3.2.2  Ground  Effect 


Ground  effects  are  non-turbulent  forces  on  the  aircraft,  which  are  experienced  at 
approximately  one  half  of  one  wingspan  in  altitude.  Ground  effects  are  very  low  in 
frequency.  In  most  modem  fighter  aircraft,  the  pilot  is  taught  to  maintain  a  constant 
angle  of  attack  during  an  approach  to  landing  and  to  ignore  any  ground  effects. 

Therefore,  ground  effects  may  not  be  considered  as  critical  a  cue  as  it  would  be  in  many 
earlier  aircraft.  However,  ground  effect  force  cue  simulation  may  be  important  in  order 
to  train  the  pilot  to  ignore  them.  Ground  effects  like  buffet  could  be  considered 
correlated  disturbances  since  it  occurs  in  a  particular  part  of  the  flight  envelope. 

3.2.3  Vibration 

Vibrations  are  higher  frequency  forces  such  as  engine  vibrations  induced  into  the 
airframe,  stores  vibrations,  and  cooling  turbine  vibrations.  Normally,  vibrations  would  be 
considered  non-correlated  disturbances.  However,  sudden  or  recognizable  changes  in 
vibration  could  provide  an  alerting  cue  and  be  considered  a  correlated  disturbance  cue. 
These  could  include  events  such  as  detection  of  a  engine  malfunction  or  confirmation  of 
an  event  such  as  firing  of  a  gun.  Vibration  can  cause  fatigue.  Under  certain  conditions,  it 
can  also  cause  some  degradation  of  a  pilot’s  visual  performance.  Time  to  interpret  a 
display  can  increase  as  a  result  of  vibrations  depending  upon  the  frequency  and  intensity. 

3.2.4  Turbulence 

Turbulence  is  a  non-regular  atmospheric  disturbance  acting  on  the  airframe.  It  is 
of  varying  frequency  and  amplitude.  Its  frequency  generally  ranges  at  or  below  that  of 
buffet.  Turbulence  may  be  classified  as  a  non-correlated  disturbance.  It  increases  pilot 
workload  and  makes  flight  tasks  such  as  target  tracking  and  flight  control  during  low 
level  high-speed  flight  more  difficult. 

3.3  Audio  Frequency  Forces 

Auditory  cues  are  sensed  by  not  only  the  pilot’s  ears,  but  by  proprioception. 

Audio  frequency  forces  overlap  with  vibration  forces.  In  the  aircraft,  the  pilot  perceives 
audio  frequency  forces  associated  with  the  aircraft  such  as  engine  noise/vibration, 
actuation  of  control  systems  and  deployment  of  weapons.  They  may  be  perceived  by  the 
ears  and/or  felt  through  the  limbs  (including  bone  conduction,  skin,  and  possibly  by  the 
chest  diaphragm).  Audio  forces  may  be  either  a  correlated  or  disturbance  cue.  Correlated 
alerting  cues  include  engine  malfunction,  gun  firing,  and  release  of  stores.  Firing  of  guns 
may  be  both  heard  and  felt,  whereas  release  of  stores  may  be  heard  and  felt  as  a  thump. 
For  the  latter,  the  pilot  may  also  feel  the  results  of  change  in  weight  of  the  aircraft.  In  a 
fighter  aircraft,  audio  cues  may  affect  workload  and  produce  stress. 
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SECTION  4  CRITERIA  FOR  DETERMINING 
FORCE  CUE  EFFECTIVENESS 


The  criteria  for  determining  the  contribution  and  effectiveness  of  force  cueing 
devices  may  include  three  factors:  (1)  pilot  behavior  in  the  simulator  as  compared  to 
behavior  in  the  aircraft,  (2)  pilot  performance  in  the  simulator  as  compared  to 
performance  in  the  aircraft  and  (3)  pilot  subjective  perceptions.  Other  factors  may  also  be 
considered,  such  as,  the  additional  stresses  which  force  cueing  adds  to  the  simulator 
environment  that  may  make  it  more  closely  resemble  the  aircraft  environment. 

Ideally,  the  criteria  used  to  determine  the  acceptability  of  the  simulator  force 
cueing  should  be  based  on  the  similarity  of  pilot  behavior  in  the  simulator  as  compared  to 
behavior  in  the  aircraft.  Since  there  is  little  behavioral  data  available  concerning 
performance  in  the  aircraft,  it  is  necessary  to  rely  on  analysis  by  experts  for  relating  the 
behavioral  data  gathered  in  the  simulator  to  what  would  be  expected  in  the  aircraft 

4.1  Pilot  Behavior 

If  the  pilot’s  behavior  in  the  simulator  is  similar  to  that  in  the  aircraft,  it  is  usually 
agreed  that  transfer  of  training  to  the  aircraft  should  occur.  This  does  not  mean  that  some 
positive  transfer  will  not  occur  in  many  cases  where  behavior  is  different.  Lack  of 
appropriate  force  cues  in  a  simulator  can  change  pilot  behavior  in  the  simulator.  Other 
differences  in  the  simulator,  such  as  workload  and  stress,  can  also  effect  behavior  and 
training.  The  Trial  Force-Eval  examined  how  measurement  of  pilot  behavior  may  be 
used  as  a  criterion  for  force  cue  evaluation.  Insight  regarding  the  effect  of  force  cues  can 
be  obtained  by  examining  behavioral  data  with  and  without  the  presence  of  force  cues. 

4.1.1  Control  Strategy 

Indications  are  that  lack  of  force  cueing  in  a  simulator  can  affect  the  way  a  pilot 
flies  a  simulator  as  compared  to  the  aircraft.  He  or  she  may  adapt  different  control 
strategies  than  those  used  in  the  aircraft  if  the  simulator  lacks  motion  or  other  force 
cueing.  This  may  result  in  relearning  the  tasks  in  the  aircraft. 

4.1.2  Eye  Scan  Response 

Without  the  appropriate  force  cues,  a  pilot  will  still  respond  to  vection  cues 
provided  by  the  visual  system.  However,  it  appears  that  the  rate  at  which  a  pilot  responds 
to  vection  without  the  appropriate  force  cues  may  lag  that  in  the  real  world  aircraft  (Boff, 
1988).  Force  cues,  although  they  may  not  duplicate  the  real  world,  may  speed  up  eye 
response  to  more  closely  resemble  eye  response  in  the  aircraft.  Measurements  could  be 
made  with  an  oculometer  to  compare  eye  movement  response  with  and  without 
appropriate  force  cues.  No  such  measurements  were  made  during  the  Trial  Force-Eval. 
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4.1.3  Pilot  Control  Input  Bandwidth 

The  rate  at  which  a  pilot  inputs  control  responses  may  be  referred  to  as  the  pilot’s 
control  bandwidth.  A  pilot  responds  to  force  cues  more  rapidly  than  to  visual  cues  alone. 
If  the  pilot  does  not  receive  adequate  force  cues,  the  rate  of  control  response  increases 
thus  reduced  bandwidth.  Therefore,  for  certain  tasks,  pilot  control  bandwidth  is  a  good 
measure  of  the  adequacy  of  the  force  cues.  During  the  Trial  Force-Eval,  pilot  bandwidth 
was  measured  and  compared  with  and  without  force  cueing. 

4.1.4  Pilot  Cue  Reliance 

Without  force  cueing,  the  pilot  will  be  forced  to  search  for  and  rely  on  other  cues. 
As  an  example,  he  or  she  may  have  to  check  on  engine  RPM  gauge  rather  than  feeling  a 
change  in  vibration  or  will  be  forced  to  check  the  g-meter  rather  than  feel  the  change  in  g- 
force.  Such  changes  in  visual  scan  patterns  relative  to  the  pilot’s  scan  pattern  in  the 
aircraft,  may  cause  the  pilot  to  learn  spurious  habit  patterns. 

4.2  Pilot  Physiological  State 

The  presence  or  absence  of  force  cueing  can  effect  the  pilot’s  physiological  state. 
This  may  include  heart  rate,  blink  rate,  respiration,  and  EEG.  Generally,  it  is  expected 
that  the  addition  of  force  cueing  will  increase  these  factors  because  of  increases  in 
workload  and  stress.  However,  there  may  be  some  cases  where  force  cueing  reduces 
workload,  which  may  reduce  the  physiological  stress  on  the  pilot. 

4.2.1  Workload  and  Stress 

A  pilot  must  deal  with  many  different  stressors  during  a  tactical  mission.  Typical 
stressors  include  electronic  combat  environment,  low  altitude  turbulence,  radio  chatter, 
etc.  Generally,  the  force  cueing  environment  in  an  aircraft  provides  increased  stress  on 
the  pilot.  This  stress  may  be  caused  by  increased  task  difficulty  resulting  from  turbulence 
and  high-g  forces.  It  could  also  be  caused  by  the  added  realism  of  the  simulation 
resulting  from  having  a  more  real  world  like  environment  when  force  cueing  is  added  to 
the  simulation.  Stress  can  have  an  immediate  effect  on  behavior  and  performance. 

4.2.2  Fatigue 

Turbulence  or  buffet  causes  an  increase  in  workload,  which  may  cause  fatigue. 
This  can  be  a  longer-term  effect  on  the  pilot  during  a  mission. 
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4.3  Pilot  Performance 


Pilot  performance  is  a  second  measure  of  force  cueing  effectiveness  in  a 
simulator.  In  many  cases,  the  difference  in  performance  with  different  force  cueing 
inputs  makes  performance  a  reliable  criterion  for  evaluating  force  cue  effectiveness. 
However,  in  other  cases,  performance  by  itself  is  not  a  reliable  measure  since  the  pilot 
may  be  using  alternate  cues  and/or  adapting  a  different  control  strategy  in  order  to 
compensate  for  a  lack  of  certain  cues  in  the  simulator.  In  such  cases,  the  performance 
results  in  the  simulator  may  be  very  similar  to  the  aircraft.  However,  from  a  training 
standpoint  this  may  be  less  than  fully  acceptable  since  the  pilot  may  have  to  relearn  tasks 
using  the  proper  cues  in  the  aircraft.  In  such  cases,  it  is  important  to  measure  behavior 
along  with  performance.  Areas  where  measurement  of  performance  appears  to  be 
appropriate  are  provided  in  the  subparagraphs.  In  some  cases,  performance  alone  as  a 
criterion  may  be  appropriate  whereas,  in  others,  behavior  along  with  performance  must  be 
measured. 

4.3.1  Tracking  Measures 

Positional,  rate  and  temporal  accuracy  are  all  good  measures  of  performance  of 
different  flight  tasks.  These  need  to  be  selectively  measured  and  compared  for  different 
force  cueing  configurations. 

4.3.2  Accuracy  of  Flight  Profile 

Differences  in  the  flight  profile  for  specific  maneuvers,  as  a  function  of 
differences  in  force  cueing  may  be  useful  in  determining  the  effect  of  force  cueing  on 
pilot  performance.  Typical  examples  include  maintaining  low  altitude  and  performing 
ridge  crossings.  Flight  profile  may  be  more  directly  interpretable  than  changes  in 
position,  rate  and  temporal  accuracy  data. 

4.3  3  Pilot  Subjective  Perceptions 

Experienced  training  system  engineers  can  gather  pilot  subjective  perceptions 
using  questionnaires  and  structured  pilot  debriefings/interviews.  For  the  Trial  Force- 
Eval,  the  pilots  completed  a  structured  questionnaire  following  an  evaluation  mission. 
Following  completion  of  the  questionnaire,  the  pilots  were  debriefed  using  the 
questionnaire. 
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SECTION  5  TRIAL  FORCE-EVAL  SYSTEM 
AND  EVALUATION 


The  Trial  Force-Eval  system  was  developed  by  the  Air  Force  Research  Laboratory 
and  Middendorf  Scientific  Services  Inc.,  which  included  all  simulator  systems  and  data 
collection  equipment.  This  system  consisted  of  the  computational  system,  software, 
cockpits,  avionics,  visual  image  generator,  visual  data  base  and  visual  display,  the  force 
cueing  hardware  and  dynamic  seat  drive  algorithms,  the  operator  control  systems  and 
displays,  and  the  instrumentation. 

5.1  Trial  Force-Eval  System  Capabilities 

5.1.1  Computational  System 

The  Trial  Force-Eval  system  used  three  Encore  RSX  computers  for  the  simulation 
executive  and  simulation  model  processing.  Each  RSX  computer  was  a  single  processor 
that  communicates  with  the  other  RSX  computers  via  a  reflective  memory  bus.  The 
simulation  executive  was  configured  to  run  synchronized  with  the  image  generation 
system  at  a  60  Hz  frame  rate.  Most  software  processes  were  run  at  the  60  Hz  rate  while 
some  of  the  slave  tasks  were  run  at  30  Hz.  The  following  table  indicates  what  simulation 
software  was  running,  rate  it  was  scheduled  to  run,  and  processor  where  each  task  was 
located. 


Table  1.  Simulation  Software 


Computer  System 

Processor  # 

Software/Process 

Rate 

Encore  RSX  (J) 

1 

Sim  Executive 

60Hz 

Hardware  I/O 

60Hz 

Wind  /  Turbulence 

60Hz 

Aircraft  #1  (F-15Model) 

60Hz 

Aircraft  #2  (Playback) 

60Hz 

IG  Driver 

60Hz 

HUD  Driver 

60Hz 

HDD  Driver 

60  Hz 

Data  Capture 

30Hz 

Encore  RSX  (G) 

1 

ALCOGS  Driver 

60Hz 

G-suit  /  Combat  EDGE 

60Hz 

Truth  Sensor 

60Hz 

Fire  Control 

60Hz 

Encore  RSX  (F) 

1 

20mm  Gun 

30Hz 

Reticle 

30Hz 
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5.1.2  Graphics  Computers 

The  Trial  Force-Eval  system  used  two  Silicon  Graphics  computers  for  graphic 
display  processing  of  the  cockpit  head  down  display  and  HUD  imagery.  These  graphic 
systems  were  driven  through  an  Ethernet  ring  that  operates  asynchronous  from  the  rest  of 
the  simulation.  Data  packets  were  updated  by  the  simulation  display  drivers,  at  the  rate  at 
which  they  were  scheduled,  then  sent  to  the  graphics  processors  for  interpretation  and  the 
graphic  displays,  then  update  as  fast  as  they  could  upon  receiving  those  data  packets. 

5.1.3  Image  Generator 

The  Trial  Force-Eval  system  used  one  Evans  &  Sutherland  model  4530  image 
generator  for  generating  and  controlling  the  out  the  window  (OTW)  visual  scene  at  the 
simulator  station.  A  Whidbey  Island  database  was  utilized  for  the  terrain  scene.  The 
simulation  was  configured  to  run  synchronously  with  the  image  generator  at  a  60  Hz 
frame  rate.  A  Mig-29  moving  model  was  used  to  represent  the  second  aircraft.  The  only 
unusual  image  generation  capability  incorporated  in  the  Trial  Force-Eval  system  was  the 
ability  to  create  a  3 -dimensional  pathway  in  the  sky  using  partial  tank  models  as  pathway 
markers.  This  implementation  was  dubbed  the  "Pylon  Course".  It  was  a  series  of 
diamond  shaped  hoops  that  the  test  aircraft  is  required  to  fly  through  at  the  proper 
orientation.  Each  hoop  was  a  set  of  five  turretless  T-72  tank  bodies.  The  hoops  were 
arranged  in  a  diamond  formation  centered  on  the  intended  flight  path  as  shown  in 
Figure  1. 


Figure  1 .  Pylon  Course  Hoop. 
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5.1.4  Visual  Display 

Original  plans  called  for  the  integration  of  the  force  cueing  devices  into  the 
facility’s  MS-1  large  dome  simulator  station;  however,  the  visual  fidelity  was  not 
sufficient  to  support  all  required  Trial  Force-Eval  tasks.  Instead,  a  rear  projection  screen 
that  had  been  set  up  for  initial  checkout  and  dynamic  seat  algorithm  development  was 
used  for  the  Trial  Force-Eval.  This  display  had  a  field-of  view  (FOV)  64  degrees  wide  by 
50  degrees  high.  The  imagery  was  rear  projected  with  a  General  Electric  light  valve 
projector  onto  a  8  by  10  foot  screen  in  front  of  the  cockpit.  HUD  graphics  were  video 
mixed  with  the  visual  scene.  This  provided  the  simulator  station  with  OTW  visual  and 
HUD  flight  data. 

5.1.5  Simulator  Cockpit  Station 

The  simulator  cockpit  station  used  for  the  Trial  Force-Eval  was  built  from  various 
simulator  components  specifically  for  the  evaluation.  The  cockpit  itself  was  built  around 
the  ALCOGS  force  cueing  seat  by  mounting  an  F-15  throttle  quadrant  on  the  left  armrest 
and  a  side  stick  with  F-15  grip  on  the  right  armrest.  The  side  stick  was  a  force  stick, 
similar  to  an  F- 16  side  stick  that  generates  stick  inputs  based  on  force  measurements  with 
very  little  motion  from  the  stick  itself.  Head  down  instruments  was  displayed  on  a  high 
resolution,  29-inch  monitor  located  directly  in  front  of  the  ALCOGS  seat.  This  monitor 
was  located  to  provide  the  pilot  with  a  1 5-degree  over-the-nose-viewing  angle  with 
respect  to  the  OTW  imagery. 

5.1.6  Simulation  Software 

The  simulation  executive  was  a  FORTRAN  program  that  allows  for  the 
scheduling  and  activation  of  all  of  the  simulation  components  across  multiple  computers 
and  processors.  The  individual  components  were  written  in  FORTRAN,  C,  C++  or  ADA. 
This  executive  allowed  the  simulation  operators  to  create  schedules  for  all  of  the 
simulation  components  based  on  whole,  even  multiples  of  the  base  simulation  frame  rate, 
which  was  16.6667  ms  (60  Hz). 

5.1.7  Simulation  Models 

The  aircraft  model  that  was  utilized  in  the  simulation  was  a  6  degree  of  freedom 
(DOF),  high  fidelity  model  of  an  F-15C.  This  model  was  written  in  FORTRAN  and 
originated  from  McDonnell  Douglas  in  support  of  past  flying  qualities  programs.  To 
reduce  the  number  of  variables  in  the  Trial  Force-Eval  tests,  the  aircraft  model  was 
configured  to  maintain  fixed  mass  properties  and  assumed  no  external  stores.  A  landing 
gear  model  was  also  integrated  with  the  F-15C  model  to  provide  the  capability  to 
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simulate  takeoffs  and  landings.  The  model  responded  to  speedbrakes,  landing  gear,  stick 
and  throttle  inputs.  Rudders  were  not  used. 

5.1.8  Environment 

The  Trial  Force-Eval  system  incorporated  atmospheric  turbulence  effects  for  some 
landing  scenarios.  The  model  used  to  generate  the  turbulence  was  a  Winds/Gusts/ 
Turbulence  model  that  originated  from  NASA  Dryden.  The  output  of  this  model  was 
incorporated  into  the  aircraft  model  aerodynamics  and  equations  of  motion  algorithms  to 
ensure  that  the  aircraft  achieved  the  appropriate  dynamic  response  to  the  atmospheric 
disturbances.  Turbulence  model  gust  lengths  and  RMS  scales  were  adjusted  to  provide 
moderate  turbulence  for  those  test  runs  that  required  this  effect. 

5.1.9  Sensors 

The  Trial  Force-Eval  system  did  not  require  realistic  sensor  models  so  a 
simplistic,  truth  data  radar  model  was  implemented.  This  model  gave  the  pilot  perfect 
radar  track  information  for  the  tracked  aircraft.  This  provided  the  pilot  the  capability  to 
designate  the  other  aircraft  as  the  primary  target  and  obtain  relative  geometry  information 
for  that  aircraft  relative  to  the  ownship.  This  information  was  displayed  on  the  HDD  and 
HUD  displays  and  only  applies  to  those  test  scenarios  that  contain  a  second  aircraft.  The 
radar  track  data  was  also  required  for  driving  the  Lead  Computed  Optical  Sight  System 
(LCOSS)  gun  reticle  during  the  Air-to-Air  gun-tracking  task. 

5.1.10  Weapons 

The  only  weapon  utilized  in  the  Trial  Force-Eval  was  a  generic  20mm  gun  model. 

5.1.11  Situation  Display 

The  Situation  Display  used  two  range  rings  centered  on  the  ownship  for  range 
reference  lines;  one  at  half  of  the  current  display  range  and  the  other  at  full  display  range. 
Indicated  range  was  measured  from  the  ownship  symbol  to  the  top  edge  of  the  display. 
Tracks  were  positioned  in  azimuth  on  the  display  along  imaginary  radial  lines  that  are 
measured  relative  to  the  ownship  longitudinal  body  axis. 

5.1.12  Weapons  Format 

The  weapons  (WPNS)  format  displayed  status  of  ownship  weapons,  fuel,  and 
countermeasures.  Digital  readouts  for  gun  rounds,  fuel  (lb.),  chaff,  and  flare  were 
provided  along  with  graphical  depictions  of  the  Short  Range  Missiles  (SRM)  and 
Medium  Range  Missiles  (MRM)  on  board  the  aircraft. 
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5.1.13  HUD 


The  HUD  symbology  was  essentially  the  same  as  the  F-15E  HUD  symbology 
with  modifications  to  reduce  clutter  and  to  meet  research  Trial  Force-Eval  needs.  MRM, 
SRM  and  Gun  modes  were  all  supported,  although  the  missile  modes  were  not  utilized  in 
these  tests.  Navigation  information  was  available  as  well  as  a  continuously  computed 
impact  point  (CCIP)  air-to-ground  bombing  sight. 

5.1.14  Flight  Controls 

The  stick  and  throttles  provide  aircraft  control  using  an  F-15C  model.  The  stick 
and  throttle  grips  were  used  to  provide  the  switchology  needed  for  control  of  the 
simulated  aircraft  systems.  They  provide  the  pilot  immediate  access  to  control  of  the 
displays  when  was  not  appropriate  to  use  the  bezel  push-button. 

5.2  Force  Cueing  Devices 

Force  cueing  devices  used  for  the  Trial  Force-Eval  were  the  Advanced  Low-Cost 
g-Cueing  System  (ALCOGS)  dynamic  seat,  anti-g  suit,  and  Combat  Edge  Positive 
Pressure  Breathing  System.  Another  device,  the  Clark  Audio  Transducer,  was  considered 
for  the  Trial  Force-Eval;  however,  due  to  problems  in  obtaining  useful  data  to  drive  the 
device,  it  was  not  included  in  the  evaluation.  Although  it  was  not  part  of  the  evaluation, 
it  is  addressed  briefly  in  this  section  of  the  report. 

5.2.1  ALCOGS  Dynamic  Seat 

The  ALCOGS  dynamic  seat  originally  developed  by  Air  Force  Human  Resources 
Laboratory  (AFHRL)  was  refurbished  and  integrated  into  the  evaluation  simulator.  The 
ALCOGS  includes  hydraulic  actuators,  which  provided  an  active  moving  seat  and  three 
thin  cushion  surface  bladders  for  localized  pressure  and  tactile  area-of-contact  stimuli 
generation.  The  seat  pan  has  passive  thigh  ramps  and  tuberosity  stimulating  blocks.  The 
seat  also  included  differential  lap  belt  drive  and  implementation  of  lower  backrest  to 
provide  strong  area-of-contact  cues  for  vertical  and  longitudinal  acceleration.  Table  5.2.1 
provides  a  synopsis  of  the  performance  characteristics  of  the  ALCOGS. 

5.2.1.1  ALGOGS  Seat  Refurbishment 

Although  the  ALCOGS  is  approximately  twenty  years  old,  it  still  represents  state- 
of-the-art  with  respect  to  performance.  Due  to  the  age  of  the  device,  a  great  deal  of 
refurbishment  and  testing  work  had  to  be  completed  before  the  seat  could  be  operated. 

For  the  Trial  Force-Eval,  the  seat  components  were  driven  in  the  following  manner: 
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(1)  Seat  Bladder  (pneumatic)  -  Positive  Sustained  g’s 

(2)  Back  Bladder  (pneumatic)  -  Surge  for  afterburner,  speedbrakes,  etc. 

(3)  Seat  Pan  Pitch  and  Roll  (hydraulic)  -  Aircraft  Velocity  and  Acceleration 

(4)  Backrest  Pitch  and  Yaw  (hydraulic)  -  Aircraft  Velocity  and  Acceleration. 

Table  2.  ALCOGS  Performance 


[  ALCOGS  Performance 

Component 

Axis 

Excursion 

Response 

Seat  Pan 

Pitch,  Roll, 

+/- 12  degrees 

36  ms,  7.3  Hz 

Heave 

+/- 1 .25  inches 

Fore- Aft 

+/- 1.0  inches 

Backrest 

Pitch 

+/-  6  degrees 

36  ms,  7.3  Hz 

Yaw 

+/-  9  degrees 

Surge 

+/- 1.0  inches 

Backrest  Bladder 

Roll 

3  psi 

<3  Hz 

Seat  Bladder 

Surge 

3  psi 

<3  Hz 

Seat  Shaker 

Heave 

+/-  0.25  inches 

34  Hz 

Lap  Belt 

Fore- Aft 

+/- 1.5  inches 

30  ms,  10  Hz 

5.2.1.2  ALCOGS  Seat  Drive  Algorithms 

The  pitch  angle  of  the  seat  pan  and  backrest  were  driven  with  a  combination  of 
aircraft  pitch  velocity  and  pitch  acceleration.  The  roll  angle  of  the  seat  was  driven  with  a 
combination  of  aircraft  roll  velocity  and  roll  acceleration.  The  yaw  angle  of  the  backrest 
was  driven  with  a  combination  of  sideslip  angle  and  sideslip  angle  rate. 

The  drive  laws  were  implemented  in  such  a  manner  that  allowed  scaling  to  be 
specified  as  maximum  seat  movement.  For  example,  the  seat  pan  can  roll  as  much  as 
twelve  degrees,  but  the  scaling  parameter  for  roll  could  be  set  to  only  five  degrees.  The 
coefficient  for  the  basic  drive  law  was  then  back  calculated  using  vehicle  performance 
information  across  a  wide  range  of  flight  conditions,  specifically,  mach  number.  Due  to 
high  frequency  effects,  pitch  acceleration  had  to  be  ramped  out  near  mach  one. 

The  seat  pan  bladder  was  driven  with  z-axis  acceleration  for  simulating  positive  g. 
The  original  software  for  this  dynamic  seat  was  written  so  that  the  bladder  pressure  was 
decreased  for  positive  g.  This  lowered  the  pilot  onto  hard  blocks  to  simulate  increased 
pressure  on  the  buttocks.  This  also  lowered  the  pilot  with  respect  to  the  rest  of  the 
cockpit.  For  the  Trial  Force-Eval,  the  drive  law  was  written  so  that  the  bladder  pressure 
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increased  with  positive  g’s  which  increased  the  hardness  of  the  seat  and  the  perceived 
force.  Although  this  approach  is  somewhat  counter  intuitive,  it  received  consistent 
approval  from  pilots  during  the  Trial  Force-Eval. 

Since  there  is  a  tendency  for  the  pilot  to  sink  in  the  seat  during  high-g 
maneuvering,  it  would  be  expected  that  there  is  a  need  to  lower  the  pilot’s  eye  height  in 
the  simulator.  This  could  have  been  done  with  the  hydraulic  actuators  in  the  seat. 
However,  the  Trial  Force-Eval  pilots  did  not  feel  this  was  necessary. 

A  similar  approach  was  used  for  the  backrest  bladder,  which  was  driven  with  x- 
axis  acceleration.  When  the  simulated  aircraft  accelerated,  the  bladder  pressure 
increased. 

5.2.2  Anti-g  Suit 

The  Trial  Force-Eval  included  an  active  anti-g  suit  as  a  force  cueing  device.  Since 
in  the  1-g  environment  of  a  simulator,  there  is  no  blood  pooling  in  the  lower  extremities 
during  high-g  maneuvering,  suit  pressurization  was  scaled  down.  Onset  inflation 
occurred  at  2  g,  with  full  inflation  to  6.5  psi  at  9  g. 

5.2.3  Combined  Advanced  Technology  Enhanced  Design  G  Ensemble  (Combat 
Edge) 


Both  the  F-15  and  the  F-16  aircraft  are  fitted  with  anti-g  Loss  of  Consciousness 
(LOC)  systems,  called  Combat  Edge,  in  order  to  extend  the  pilot’s  g-tolerance.  Figure  2 
provides  a  sketch  of  Combat  Edge.  Combat  Edge  is  designed  to  provide  tactical  aircrew 
members  with  a  positive  pressure  breathing  system  for  positive  vertical  acceleration  (+4 
to  +9  g)  and  protection  at  altitudes  up  to  50,000  feet.  The  system  components  consist  of 
modified  helmet,  oxygen  mask,  vest  assembly,  breathing  regulator  and  required 
integration  and  connection  hardware.  The  Combat  Edge  equipment  is  used  in 
conjunction  with  a  standard  Air  Force  anti-g  cutaway  suit.  In  the  aircraft,  the  anti  g- valve 
senses  the  presence  of  positive  g  conditions  and  provides  pressurized  engine  bleed  air  at  a 
predetermined  schedule  to  the  anti  g-suit  and  the  oxygen  regulator  pressure  sensor. 
Regulated  air  and  oxygen  at  a  predetermined  schedule  is  supplied  to  the  Combat  Edge 
equipment  to  provide  vest  inflation  and  positive  pressure  breathing  gas  (PPBG)  for 
breathing.  The  vest  is  worn  on  the  upper  torso  of  the  crewmember  over  the  standard 
flight  suit  and  provides  chest  counter  pressure  during  PPBG  use.  The  Combat  Edge 
helmet  includes  a  modification  whereby  an  inflatable  bladder  is  installed  to  provide  the 
automatic  mask  tensioning  required  for  PPBG.  All  elements  of  Combat  Edge,  which  may 
conceivably  provide  useful  cues,  were  activated.  Similar  to  the  anti  g-suit.  Combat  Edge 
required  rescaling  in  order  to  safely  operate  in  the  one-g  environment  of  the  flight 
simulator.  Normally,  the  anti-g  suit  pressure  controls  Combat  Edge.  Because  the  g-suit 
pressure  was  scaled  down,  a  separate  electronically  controlled  g-valve  was  needed  to 
drive  Combat  Edge. 
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approximately  8  inches  in  diameter  and  2  inches  high  and  is  mounted  on  a  single  3/8-inch 
threaded  stud.  The  device  is  capable  of  accepting  200  watts  RMS  power  with  a 
transduction  force  of  1 .2  pounds  per  watt.  The  frequency  range  is  from  5Hz  to  200khz. 
The  transducer  is  most  effective  with  a  highly  rigid  mounting  structure.  It  appears  that 
only  one  transducer  may  be  necessary  in  a  simulator. 

This  device  should  support  a  variety  of  forces  which  are  both  heard  and  felt 
including:  (1)  engine  vibration,  (2)  actuation  of  systems,  (3)  firing  of  guns,  (4)  release  of 
stores,  and  (5)  background  audio  such  as  radio  chatter. 

5.2.4. 1  Clark  Audio  Transducer  Integration 

Installation  of  the  Clark  Transducer  is  straightforward.  It  mounts  to  any  surface 
with  a  single  threaded  stud.  However,  it  was  quickly  learned  that  driving  the  device  in  a 
manner  that  takes  full  advantage  of  its  capabilities  for  this  application  is  not  so 
straightforward.  This  transducer  will  provide  a  useful  output  down  to  as  low  as  5  Hz. 

For  simulation  of  many  of  the  important  cues  associated  with  an  aircraft,  such  low 
frequency  response  is  important.  However,  standard  audio  amplifiers  do  not  function  at 
such  a  low  frequency.  Also,  recorded  aircraft  data  that  was  available  to  the  program  was 
recorded  using  audio  recording  equipment  that  apparently  did  not  function  well  at  such 
low  frequencies.  Attempts  to  compensate  for  the  shortcomings  of  the  recorded  data  by 
modification  were  not  successful.  A  high  power  linear  amplifier  was  obtained  which 
drove  the  transducer  effectively  at  the  lower  frequencies;  however,  the  audio  data 
problem  was  never  resolved.  Therefore,  the  Clark  Transducer  was  not  included  in  the 
Trial  Force-Eval. 

5.3  Trial  Force-Eval  Instrumentation 

Most  of  the  requirements  for  data  that  needed  to  be  recorded  could  be  determined 
intuitively.  These  requirements  were  determined  with  consultation  with  expert 
engineering  pilots.  When  there  was  a  question  as  to  the  potential  need  it  was  usually 
decided  in  favor  of  recording  data  until  it  could  be  shown  that  the  data  was  of  no  value. 
The  following  data  were  selected  for  recording  in  the  Trial  Force-Eval: 

1)  Flight  control  and  throttle  inputs  by  evaluation  pilot  (both  magnitude  and  rate). 

2)  State  vectors  for  each  aircraft. 

-  Position 

-  Linear  and  angular  rates  and  accelerations  for  each  aircraft. 

=  Related  to  cockpit  for  evaluation  cockpit  (includes  g,  g-rate). 

=  Related  to  either  cockpit,  center  of  gravity  or  center  of  rotation 
for  other  cockpit. 
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3)  Relative  linear  and  angular  positions  and  rates  between  aircraft  with  evaluation 
cockpit  as  a  reference. 

4)  Commanded  Combat  Edge  pressure. 

5)  Commanded  g-suit  pressure. 

6)  Indicated  airspeed/Mach. 

7)  Indicated  and  Radar  altitude. 

8)  Speedbrakes:  in/out. 

9)  Landing  gear:  up/down. 

10)  Pickle  button  or  trigger  depression  as  well  as  selected  switch  positions. 

1 1)  Dynamic  seat  commanded  and  recorded  positions. 

12)  ALCOGS  commanded  bladder  pressures. 

5.4  Evaluation  Task  Scenarios 

Task  scenarios  were  chosen  which  were  potentially  sensitive  to  force  cueing. 

They  all  involved  high  task  loading.  With  the  exception  of  the  pylon  course,  task 
scenarios  were  representative  of  different  portions  of  a  realistic  mission. 

5.4.1  Landing  Task 

A  landing  task  was  used  to  evaluate  the  effectiveness  of  angular  seat  cueing  on  the 
pilot’s  performance  during  approach  and  touchdown.  A  moderate  to  severe  turbulence 
was  present  on  one-half  of  the  trials.  Landings  were  made  with  and  without  force  cueing. 
Because  of  the  low-g  nature  of  the  task,  the  only  active  cueing  device  during  landing  was 
the  dynamic  seat.  Initial  conditions  were:  altitude  1500  feet  AGL,  aircraft  8  miles  out,  on 
centerline,  airspeed  193  knots,  speed  brake  out  and  gear  down.  The  waypoint  marker  was 
overlaid  on  the  runway  to  indicate  the  approach  point,  which  is  1000  feet  down  the 
runway. 

The  pilots  were  instructed  to  decrease  airspeed  to  155  knots,  begin  descent  at  five 
miles,  and  maintain  a  three-degree  glideslope.  The  pilots  were  told  to  attempt  a  minimal 
sink  rate  landing.  The  actual  touchdown  point  could  be  further  down  the  runway  than  the 
1000  feet  approach  point.  The  pilots  were  also  instructed  to  maintain  a  wings-level 
approach  while  on  centerline,  specifically  in  the  presence  of  turbulence.  Pilots  were 
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given  two  practice  trials,  one  with  and  one  without  turbulence.  Two  sessions  of  eight 
landing  each  were  performed.  The  order  of  the  independent  variables  (cueing  and 
turbulence)  was  counter  balanced. 

5.4.2  Low  Level  Flight  With  Weapon  Delivery  Task 

A  low  level  task  with  waypoints  was  used  to  evaluate  the  usefulness  of  force 
cueing  during  low-level  flight  and  weapon  delivery.  This  mission  was  flown  with  and 
without  force  cueing.  The  force  cueing  included  the  dynamic  seat,  with  bladders.  Combat 
Edge  and  anti  g-suit.  The  initial  conditions  were:  altitude  1000  feet,  airspeed  475  knots, 
speed  brake  in,  gear  up,  heading  074  degrees  and  the  aircraft  25  miles  from  waypoint  one. 
The  pilots  were  instructed  to  fly  at  the  lowest  comfortable  altitude  and  maneuver  for 
optimal  ridge  crossing  until  they  reach  waypoint  two.  This  delineates  the  terrain 
following  segment  of  the  mission.  After  waypoint  two,  the  pilots  performed  an  offset 
pop-up  weapon  delivery  task  using  a  CCIP  pipper  (continuously  computed  impact  point). 
The  pilots  were  instructed  to  perform  all  four-g  turns.  Pilots  were  given  one  practice 
trial.  Two  data  collection  sessions  of  four  missions  each  were  performed.  The  order  of 
the  independent  variable  (cueing)  was  counter  balanced. 

5.4.3  Air-to-Air  Guns  Task 

The  air-to-air  guns  task  was  a  medium  g-level  task  which  employed  a  tracking 
requirement.  This  scenario  was  flown  with  and  without  force  cueing.  The  force  cueing 
included  the  dynamic  seat,  with  bladders,  Combat  Edge,  and  anti  g-suit.  The  initial 
conditions  were:  altitude  10,000  feet,  airspeed  448  knots,  speed  brake  in,  gear  up  and 
target  aircraft  1000  feet  ahead.  The  avionics  included  a  lead-computing  optical  sight 
system  (LCOSS).  The  pilots  were  instructed  to  keep  the  LCOSS  pipper  on  the  target  and 
maintain  a  distance  of  1000  feet.  Distance  to  the  target  and  closure  rate  was  indicated  on 
the  LCOSS  reticule.  Pilots  were  given  one  practice  trial.  Two  data  collection  sessions  of 
four  missions  each  were  performed.  The  order  of  the  independent  variable  (cueing)  was 
counter  balanced. 

5.4.4  Pylon  Course  Task 

The  pylon  course  task  consisted  of  a  “highway  in  the  sky”  which  was  indicated  by 
sets  of  pylons  that  were  spaced  five  seconds  apart.  The  location  of  the  pylons  was  based 
on  a  pre-recorded  flight  by  an  experienced  pilot  performing  a  box  maneuver.  The  g- 
loading  necessary  to  accurately  fly  the  course  increased  with  time.  Specifically,  the  end 
of  the  course  was  much  harder  to  fly  than  the  beginning.  The  orientation  of  the  pylons 
indicated  the  roll  attitude  from  the  pre-recorded  flight.  A  lead  aircraft  flew  the  course  just 
ahead  of  the  pilot’s  simulated  aircraft.  The  force  cueing  included  the  dynamic  seat,  with 
bladders,  Combat  Edge,  and  anti  g-suit.  The  initial  conditions  were:  altitude  1 0,000  feet. 
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airspeed  333  knots,  speed  brake  in  and  gear  up  with  the  lead  aircraft  1500  feet  ahead  and 
accelerating.  The  pilots  were  instructed  to  follow  the  lead  aircraft  through  the  pylon 
course  and  match  the  roll  attitude  of  the  pylons.  The  pilots  were  given  one  practice  trial. 
Two  data  collection  sessions  of  four  missions  each  were  performed.  The  order  of  the 
independent  variable  (cueing)  was  counter  balanced. 

5.4.5  Formation  Flight  Task 

A  formation  flight  task  was  a  high-gain  task  which  required  continual  pilot  input. 
The  limited  field-of-view  visual  simulation  display  increased  the  task  difficulty.  The 
flight  path  of  the  lead  aircraft  was  based  on  a  pre-recorded  flight.  Similar  to  the  pylon 
course,  the  g  loading  increased  with  time.  Force  cueing  included  the  dynamic  seat,  with 
bladders,  Combat  Edge,  and  anti  g-suit.  The  initial  conditions  were:  altitude  10,000  feet, 
airspeed  333  knots,  speed  brake  in  and  gear  up  with  the  trailing  aircraft  13  feet  down  and 
60  feet  behind  the  lead  aircraft.  The  pilots  were  told  to  maintain  the  initial  relative 
position  throughout  the  flight.  The  pilots  were  given  specific  references  on  the  display  to 
aid  them  in  maintaining  the  relative  position.  These  references  used  the  burner  cans  of 
the  lead  aircraft  and  the  HUD  symbology  on  the  pilot’s  display.  Pilots  were  given  one 
practice  trial.  Two  data  collection  sessions  of  four  missions  each  were  performed.  The 
order  of  the  independent  variable  (cueing)  was  counter  balanced. 

Pilots  participated  in  two  data  collection  sessions.  All  five  of  the  task  scenarios 
were  performed  during  each  session.  Four  Trials  (two  with  cueing,  two  without)  were 
conducted  for  each  scenario,  except  landing  which  required  eight  trials  due  to  the 
turbulence/no  turbulence  condition.  Forty-eight  trials  were  conducted  for  each  pilot. 

5.5  Data  Analysis 

Performance  data,  control  behavior  and  subjective  data  were  analyzed  to  identify 
differences  between  the  cueing  and  no-cueing  conditions.  Some  trials  had  to  be 
eliminated  to  remove  outliers.  These  resulted  from  system  failure,  pilot  error,  and 
operator  error.  Pilot  error  included  mistakes  such  as  loosing  the  pylon  course  and  not 
relocating  it.  In  another  case,  the  pilot  increased  the  following  distance  in  the  formation 
flight  task  to  make  the  task  easier.  Two  trials  were  lost  due  to  hardware  failure  on  the 
visual  system.  Three  trials  were  lost  due  to  operator  error. 

Some  tasks  were  divided  into  segments  and  the  data  were  analyzed  separately. 
The  waypoint  task  was  divided  into  a  terrain  following  segment  and  a  weapon  delivery 
segment.  The  pylon  data  were  divided  into  two  segments  where  the  difficulty  level 
increased.  The  first  three  miles  on  the  landing  approach  where  the  vehicle  was  in  level 
fight  were  not  included  in  the  analysis. 

Descriptive  statistics  (mean,  standard  deviation,  and  RMS)  were  computed  for 
several  aircraft  state  variables  to  investigate  differences  between  cueing  and  no-cueing. 
Power  spectral  analysis  was  performed  on  stick  activity  to  identify  differences  in  control 
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behavior  with  and  without  cueing.  Subjective  debriefing  questionnaires  were  reviewed 
and  summarized. 

5.6  Pilots 

All  of  the  pilots  used  in  the  Trial  Force-Eval  had  at  least  2000  hours  in  modem 
fighter  aircraft.  Additionally,  all  four  have  served  as  instructor  pilots  and  three  of  the 
four  are  still  active.  The  four  pilots  had  experience  in  the  five  flight  scenarios  used  in  the 
study.  The  pilots  were  given  written  instructions  before  performing  each  scenario  to 
ensure  consistency.  A  checklist  was  used  prior  to  each  test  session  to  make  sure  all 
switches  and  cueing  hardware  were  in  the  correct  configuration. 
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SECTION  6  RESULTS  OF  THE  TRIAL  FORCE-EVAL 


6.1  Summary  of  Pilot  Behavior  Evaluation 

The  results  of  the  evaluation  of  the  various  tasks  flown  with  and  without  force 
cueing  are  summarized  below: 

6.1.1  Landing  Task 

The  pilots  were  instructed  to  maintain  a  wings-level  approach  when  on  centerline. 
Previous  drive  law  research  (McMillan,  Cress,  etc.,  1990)  shows  that  the  dynamic  seat  is 
very  effective  for  improving  this  type  of  attitude  maintenance  in  the  presence  of 
turbulence.  It  was  evident  from  the  data  that  the  four  pilots  did  benefit  from  the  presence 
of  force  cueing.  However,  the  benefit  was  much  smaller  than  expected.  The  pilots  were 
also  told  to  land  on  centerline.  There  was  an  improvement  in  this  measure  with  force 
cueing  present.  This  was  most  likely  the  result  of  the  tighter  approach  noted  above. 

There  was  an  interesting,  and  unwanted,  effect  discovered  in  the  stick  data.  With 
the  turbulence  turned  off,  there  was  higher  stick  activity  when  force  cueing  was  present 
(figure  3).  This  effect  was  consistent  for  all  four  pilots.  This  suggests  that  there  was 
some  degree  of  biomechanical  coupling,  which  could  slightly  degrade  the  performance 
benefit  of  the  seat.  The  coupling  was  most  likely  caused  by  the  fact  that  the  stick  was 
rigidly  mounted  (i.e.,  it  did  not  move  in  synchrony  with  the  seat).  Therefore,  seat  motion 
resulted  in  body  motion,  which  coupled  into  the  stick.  Another  possible  explanation  for 
this  stick  activity  may  be  that  when  force  cueing  is  present,  the  pilot  is  continually 
sampling  for  feedback  by  providing  small  stick  inputs. 

6.1.2  Pylon  Course  Task 

There  were  substantial  benefits  of  force  cueing  in  the  pylon  course  task.  When 
cueing  was  present,  the  pilots  flew  the  course  more  accurately  (figure  4),  with  less  stick 
activity  (figure  5)  and  more  realistic  stick  activity.  Specifically,  the  z-axis  g  loading  had 
considerably  less  variance  when  cueing  was  turned  on.  Examination  of  g  profile  plots 
indicated  that  the  pilots  were  putting  in  more  g  commands  that  were  erratic  when  cueing 
was  off.  With  cueing  present,  the  pilots  commanded  more  deliberate,  and  consistent,  z- 
axis  loading. 

It  is  reasonable  to  suggest  that  the  more  consistent  g  loading  is  attributable  to  the 
anti  g-suit  and  Combat  Edge  activation.  The  presence  of  seat  motion  also  seems  to  be 
helping  the  pilots  in  the  pylon  task.  The  variance  in  the  vehicle’s  angular  rates  is  much 
lower  when  cueing  is  present.  This  enabled  the  pilots  to  fly  the  course  more  accurately, 
which  is  indicated  in  the  relative  geometry  measures  (relative  azimuth  and  elevation 
between  the  lead  aircraft  and  the  trailing  aircraft). 
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Figure  4.  Pylon  Course  Task.  Pilots  followed  the  lead  aircraft  through  the  pylon  course 
more  accurately  when  force  cueing  was  fumed  on.  A  similar  trend  was  present  in  relative 
azimuth.  Standard  deviation  is  used  because  of  a  large  mean  due  to  the  following 
distance. 
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Segment 

Figure  5.  Stick  Activity.  When  force  cueing  was  present,  the  pilots  used  less  stick 
activity  to  fly  through  the  pylon  course.  The  data  were  divided  into  two  segments  at  the 
point  where  the  difficulty  level  increased. 


6.1.3  Air-to-Air  Guns  Task 

Air-to-air  guns  tracking  task  results  were  similar  to  the  results  experienced  with 
the  pylon,  however,  differences  between  cueing  and  no  cueing  are  not  as  pronounced. 
This  is  most  likely  because  the  air-to-air  guns  task  requires  less  g-loading  than  the  pylons 
task.  On  average,  the  pilots  did  a  better  job  on  keeping  the  pipper  on  the  target  when 
force  cueing  was  present. 

6.1.4  Low  Level  Flight  Task 

Other  than  a  reduction  in  stick  activity,  the  low-level  task  with  waypoints  showed 
little  difference  between  the  cueing  conditions  during  the  terrain  following  segment. 
There  are  two  reasons  for  this.  First,  the  task  was  not  demanding  in  terms  of  g  loading. 
Three  of  the  four  pilots  rarely  pulled  more  than  two  g’s  for  this  segment.  Second,  the 
instructions  to  the  pilots  led  to  different  control  strategies.  The  pilots  were  told  to  fly  at 
their  lowest  comfortable  altitude.  While  one  pilot  was  extremely  aggressive  in  following 
the  terrain,  another  simply  skimmed  over  the  valleys. 

6.1.5  Formation  Flight  Task 

The  main  effect  of  cueing  on  the  formation  flight  task  was  seen  in  the  tracking- 
type  measures.  These  measures  include  the  relative  azimuth  position  off  the  nose, 
relative  elevation  position  off  the  nose,  and  the  relative  range  (figure  6).  The  pilots  did  a 
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much  better  job  of  maintaining  the  relative  position  between  the  two  aircraft  when  the 
cueing  was  present.  The  improvement  in  relative  range  may  be  a  result  of  the  thrust 
feedback  provided  by  the  backrest  bladder.  Additional  tests  are  required  to  confirm  this 
suggestion. 


Figure  6.  Formation  Flight  Task.  The  pilots  maintained  a  closer  relative  range  when 
force  cueing  was  active  during  formation  flight.  The  backrest  bladder  provided  thrust 
feedback. 


6.2  Power  Spectral  Analysis  of  Stick  Activity 

In  the  no-cueing  condition,  there  was  consistent  increased  power  in  the  one  hertz 
region  on  both  lateral  and  longitudinal  stick  activities.  This  effect  was  consistent  across 
the  five  tasks,  but  was  most  pronounced  in  the  formation  flight  task  (figure  7).  This  is  not 
surprising  given  the  high  gain  nature  of  this  task.  As  reported  earlier,  the  pilots 
performed  this  task  better  (relative  geometry)  when  force  cueing  was  turned  on. 

Therefore,  the  extra  power  in  the  one  hertz  region  was  not  needed  to  perform  the  task, 
and  consequently  represents  remnant  and/or  artifact.  The  most  likely  cause  of  this 
unneeded  stick  activity  is  due  to  pilots  correcting  for  over  shoot.  The  dynamic  seat 
provided  lead  information,  which  reduced  the  likelihood  of  over  shoot. 

6.3  Summary  of  Pilot  Debriefing 

When  asked  “How  did  the  presence  of  force  cueing  effect  your  ability  to  perform 
the  task?”,  the  pilots  agreed  that  the  cueing  allowed  them  to  perform  the  task  in  a  manner 
closer  to  how  they  would  perform  it  in  the  aircraft.  The  pilots  commented  that  the  cueing 
provided  better/earlier  feedback  on  the  effects  of  flight  control  inputs.  The  cueing 
allowed  the  pilots  to  make  finer  corrections. 
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Figure  7.  Stick  Power  Spectral  Analysis.  The  average  power  spectrum  for  the  four  pilots 
in  the  formation  flight  task  shows  increased  power  in  the  one-hertz  region  when  force 
cueing  is  turned  off. 

When  asked  “Did  the  force  cues  enhance  the  operational  realism  of  the 
simulator?”,  there  was  a  strong  consensus  to  the  affirmative.  In  fact,  one  pilot  stated,  “it 
was  the  closest  simulator  he  has  flown  and  “feels”  like  real  flying.”  Pilots  were  asked  if 
there  is  a  force  cueing  subsystem  that  could  be  added  that  might  improve  the  operational 
realism  of  the  simulator.  Comments  varied  from  no,  to  maybe,  and  that  the  increase  in 
“feel”  would  not  be  worth  the  extra  cost  and  complexity.  Limb  loading,  aural  cueing  and 
more  longitudinal  acceleration  were  mentioned  as  possibilities. 

Other  comments  included:  this  technology  could  get  more  pilots  to  agree  that  simulators 
are  getting  very  close  to  simulating  actual  flight  conditions.  If  this  technology  is  cost 
effective,  it  should  be  added  to  all  training  simulators. 

6.4  Trial  Force-Eval  Shortfalls 

The  results  of  the  Trial  Force-Eval  indicate  the  need  for  improvements  in  the 
evaluation  process  for  any  future  work.  These  improvements  range  from  simple  things 
such  as  instructions  given  to  the  pilots  to  more  complex  issues  such  as  adaptive  drive 
algorithms.  For  example,  the  instructions  for  the  low  level  flight  task  need  to  be  more 
definitive  so  that  the  pilots  will  fly  in  a  similar  manner.  In  the  landing  task,  the 
differences  between  the  cueing  conditions  were  much  smaller  than  anticipated  based  on 
previous  drive  law  research.  This  is  most  likely  due  to  minimal  dynamic  seat  cueing, 
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which  was  a  result  of  the  drive  algorithms  that  were  normalized  across  tasks  and  mach 
numbers.  A  follow-on  study  is  needed  to  evaluate  the  utility  of  drive  laws  that  are 
adaptive  to  the  task. 

Some  of  the  tasks  would  benefit  from  a  larger  field  of  view  (FOV)  visual  display. 
The  limited  FOV  used  in  this  study  caused  some  trials  to  be  missed  in  the  pylons  and  air- 
to-air  gun  tasks  because  the  pylons  (or  target)  would  go  off  the  screen  and  the  pilots  could 
not  recover.  The  limited  FOV  also  imposed  constraints  on  the  formation  flight  task.  The 
lead  aircraft  had  to  be  set  up  in  a  position  similar  to  aerial  refueling  rather  than  the  typical 
wingman  arrangement. 

Other  implications  for  a  follow  up  study  include  removal  of  any  biomechanical 
coupling,  changes  to  the  turbulence  condition  in  landing,  selection  of  an  optimum  low 
level  flight  course,  and  the  addition  of  other  force  cueing  devices.  It  may  also  be  useful 
to  perform  workload  and  physiological  recordings  and  eye  tracking  recordings. 

6.5  Observations 

6.5.1  ALCOGS  Seat  Pan  Inflation 

To  simulate  positive  g  forces,  the  ALCOGS  was  designed  to  deflate  the 
pneumatic  seat  pad.  This  would  lower  the  pilot’s  buttocks  on  to  a  firm  surface,  which 
would  increase  the  pressure  on  the  buttocks  thus  providing  the  sensation  to  the  pilot  of 
increased  g  forces.  At  the  same  time,  it  would  reduce  the  pilot’s  eye  height  as  if  he  or  she 
was  being  pushed  down  in  the  seat. 

During  the  process  of  algorithm  development,  engineering  pilots  did  not 
consistently  have  the  perception  of  increased  buttocks  pressure  by  deflating  the  bladder 
and  being  lowered  onto  the  tuberoscity  blocks.  Although  the  tuberoscities  are  the  most 
sensitive  part  of  the  buttock,  it  is  not  clear  that  there  was  an  actual  increase  in  pressure. 
There  could  be  various  reasons  for  this  method  not  consistently  providing  the  proper 
cues.  One  problem  may  be  differences  in  size  and  shape  of  pilot  buttocks  or  “one  size 
does  not  fit  all”.  This  may  be  consistent  with  what  had  been  experienced  at  Cranfield 
(Matthews,  1978) 

It  was  found  that  by  inflating  the  seat  pad,  a  reliable  sensation  of  increased 
pressure  associated  with  positive  g  force  could  be  provided.  Also,  it  would  be  expected 
that  it  would  be  necessary  to  lower  the  seat  pan  in  order  to  reduce  pilot  eye  height  during 
a  high-g  maneuver.  However,  the  engineering  pilots  did  not  feel  it  was  necessary, 
although  it  could  easily  have  been  done  with  the  seat  hydraulics. 

6.5.2  Biomechanical  Coupling 

The  movement  of  the  seat  causes  relative  motion  between  the  pilot  and  the  cock¬ 
pit  controls.  This  relative  motion  may  cause  unwanted  biomechanical  coupling.  A  simu¬ 
lator  with  a  force  stick  such  as  the  F- 16  may  be  especially  vulnerable  to  such  coupling. 
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SECTION  7  CONCLUSIONS  AND  RECOMMENDATIONS 


7.1  Conclusions 

Evaluating  force  cueing  must  be  performed  in  a  mission  context.  Evaluating 
cueing  at  the  subtask  level  may  prove  to  be  invalid  when  the  full  context  of  the  task  is 
added.  As  an  example,  during  the  Trial  Force-Eval,  it  was  learned  that  algorithms 
developed  earlier  had  only  been  validated  for  one-g  pursuit.  In  reality  pursuit  tracking 
involves  high-g  maneuvering  in  which  case  these  algorithms  did  not  work  properly. 
Consequently,  new  algorithms  had  to  be  developed. 

The  Trial  Force-Eval  provides  a  documented  method  for  future  force-cue 
evaluations.  The  combination  of  measured  pilot  behavior  and  performance  while 
performing  specific  tactical  mission  tasks  together  with  subjective  pilot  feedback 
provides  an  effective  means  of  evaluating  the  contribution  of  force  cueing  to  mission 
performance. 

Although  the  evaluation  method  does  not  address  how  transfer  of  training  may  be 
affected  by  force  cueing,  this  may  not  be  the  real  issue.  Rather  the  issue  may  be  whether 
force  cueing  will  cause  the  pilot  to  perform  and  behave  in  the  simulator  similar  to  the 
aircraft.  Behavior  in  a  simulator,  which  is  far  different  than  the  aircraft,  may  require 
relearning  in  the  aircraft. 

The  intent  of  this  program  was  to  develop  method  for  performing  simulator  force 
cueing  evaluation.  The  Trial  Force-Eval  reported  here  does  not,  in  itself,  fully  define  the 
method.  However,  based  on  the  Trial  Force-Eval,  the  following  questions  should  be 
asked  about  a  force  cueing  system  under  evaluation: 

(1)  Does  the  system  reduce  unneeded  stick  activity  in  high  gain  tasks?  Does  it 
improve  pilot  control  bandwidth? 

(2)  Without  force  cueing  in  a  simulator,  pilots  tend  to  pull  an  unrealistic  level  of 
positive  g  forces  and  in  some  case  even  negative  Gs.  Do  force  cueing  devices 
encourage  g-profiles  that  are  more  realistic? 

(3)  Does  the  system  provide  pitch-axis  lead  information  for  high  gain  pursuit-type 
tasks  (air-to-air  guns)? 

4)  A  dynamic  seat  can  provide  a  longitudinal  acceleration  cues  through  the  back 
of  the  seat.  Does  the  system  under  evaluation  provide  thrust  feedback  (i.e., 
formation  flight)? 
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7.2  Recommendations 


The  work  accomplished  under  this  effort  provides  a  method  for  conducting  force 
evaluations  for  tactical  combat  flight  simulation.  Although  this  effort  did  demonstrate  the 
positive  effects  of  a  dynamic  seat  on  pilot  behavior  and  performance,  it  did  not  address 
the  impact  of  other  force  inputs  such  as  vestibular.  It  is,  therefore,  recommended  that  a 
force  cueing  evaluation  be  conducted  on  a  simulator,  which  includes  a  full  spectrum  of 
force  cueing  devices  such  as  the  NLR  facility  at  Amsterdam.  This  simulator  includes  a 
high  performance  platform  motion,  a  dynamic  seat,  helmet  loader  and  a  full  field  of 
regard  visual  system. 

The  force  cue  drive  laws  need  to  be  reviewed  to  improve  cueing  effectiveness. 
Because  pilots  can  have  very  different  control  strategies  for  different  mission  segments, 
the  gain  of  the  algorithms  may  need  to  be  adaptive.  Additionally,  the  magnitude  of  seat 
and/or  platform  motion  may  need  to  adapt  to  the  task. 

The  only  true  method  for  validating  pilot  behavior  in  the  simulator  is  to  compare 
the  behavior  in  the  simulator  with  behavior  in  the  aircraft  while  performing  the  same  set 
of  maneuvers.  Therefore,  it  is  highly  recommended  that  that  a  flight  test  program  be 
initiated  to  collect  pilot  behavioral  data.  The  Air  Force  Research  Laboratory  (AFRL) 
Variable  Stability  In-flight  Simulator  Test  Aircraft  (VISTA)  offers  a  unique  capability  to 
conduct  flight  test  to  measure  control  behavior  in  the  aircraft.  This  is  an  F-16C  Block  40 
aircraft  which  has  been  modified  to  act  as  a  flying  test  bed.  This  aircraft  has  imbedded 
the  instrumentation  to  make  the  required  measurements. 

The  Clark  Transducer  offers  a  relatively  inexpensive  method  of  providing  cueing 
in  the  frequency  range  greater  than  five  hertz.  Originally,  this  cueing  device  was  to  be 
included  in  the  Trial  Force-Eval.  However,  it  was  found  that  to  effectively  use  this 
device,  high  fidelity  aircraft  data  in  the  five  hertz  and  higher  frequency  range  and  a  wide 
band  high  power  linear  drive  amplifier  are  required.  This  data  were  unavailable.  This 
would  best  be  obtained  by  making  digital  recording  in  a  fighter  aircraft  using  the 
transducer  as  a  pickup  device.  It  is  recommended  that  an  evaluation  of  the  transducer  be 
conducted  in  a  simulator  using  suitable  aircraft  data. 
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APPENDIX  A 


RECOMMENDED  FORCE  EVALUATION  SYSTEM 
A-1.0  Evaluation  Structure 

The  success  of  an  evaluation  will  largely  depend  on  the  structure  of  the  evaluation 
system.  The  evaluation  system  included  things  other  than  the  evaluation  hardware, 
software  and  instrumentation.  These  other  factors  include  the  mission  and  task  structure 
and  their  associated  effect  on  task  loading,  pilot  background,  evaluation  environment  and 
the  structure  of  the  evaluation  procedures  and  how  they  are  conducted. 

A-l.l  Mission  Oriented 

An  effective  evaluation  program  can  best  be  conducted  with  simulated  missions 
which  are  representative  of  what  might  be  expected  in  the  real  world  aircraft.  If  the  task 
loading  is  too  low,  the  pilot  may  have  more  time  to  interpret  flight  vehicle  state 
information  than  would  be  available  in  the  aircraft.  Therefore,  it  is  essential  that  the 
evaluation  be  carried  out  in  a  task  loaded  mission  environment  similar  to  what  is 
experienced  in  the  aircraft. 

A-1.2  Experienced  Fighter  Pilots 

Highly  experienced  operational  pilots  need  to  be  used  to  conduct  an  evaluation. 
Prior  to  the  start  of  a  formal  evaluation,  experienced  training  system  pilots  with  a  fighter 
aircraft  background  were  used  to  assist  in  development  and  to  conduct  preliminary 
evaluations  of  the  force  cueing  algorithms,  evaluation  missions,  tasks,  data  collection 
equipment  and  collection  procedures. 

A-1.3  Controlled  Evaluation  Environment 

For  an  evaluation  it  is  essential  that  the  same  simulator  configuration  be 
maintained  throughout  an  evaluation.  However,  prior  to  a  formal  Force-Eval,  it  is 
necessary  to  change  the  configuration  of  the  evaluation  system  in  order  to  look  at  the 
effects  of  differences  in  algorithms,  types  of  force  cueing,  changes  in  instrumentation, 
changes  in  mission  tasks,  and  changes  in  evaluation  procedure.  Such  changes  are 
essential  to  optimizing  the  evaluation  procedures/method. 

A-1.4  Evaluation  Tasks/Sub  tasks 

The  evaluation  should  be  conducted  in  a  mission  environment.  Specific  task 
segments,  which  appear  to  be  sensitive  to  the  force  environment,  need  to  be  identified  and 
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later  updated.  These  subtasks  should  be  closely  monitored  during  the  evaluation  for  the 
effect  that  different  force  cue  inputs  may  have  on  the  pilot’s  behavior  and  performance. 

A-2.0  Evaluation  Facility 

A-2.1  Facility  Data  Recording  Capabilities 

The  simulation  facility  should  have  the  capability  to  record  in  real-time  the 
following  types  of  data: 

1)  Flight  control  and  throttle  inputs  by  evaluation  pilot  (both  magnitude  and 
rates). 

2)  State  vectors  for  each  aircraft. 

3)  Linear  and  angular  rates  and  accelerations  for  each  aircraft. 

-  Related  to  the  cockpit  for  the  evaluation  cockpit  (include  g,  g-rate). 

-  Related  to  either  cockpit,  center  of  gravity  or  center  of  rotation  for  other 
cockpit. 

4)  Relative  linear  and  angular  position  and  rates  between  aircraft  with  evaluation 
cockpit  as  a  reference. 

5)  Evaluation  pilot  look  angles  to  other  aircraft  or  target  (head  and  eye  angles)  or 
target/selected  reference/other  aircraft  angles  and  rate  of  closure. 

6)  G-suit  pressure  or  commanded  pressure. 

7)  Evaluation  of  pilot  head  and  eye  tracking. 

8)  Pilot  physiological  state  data  (heart,  respiration,  EEG  (brain  wave),  etc.). 

9)  Indicated  airspeed/Mach. 

10)  Indicated  and  Radar  altitude. 

11)  Speedbrakes  in/out. 

12)  Pickle  button  or  trigger  depression  as  well  as  selected  switch  positions. 

13)  Force  cueing  device  movement,  acceleration  and  pressure,  and  lap  belt 
tension. 
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14)  Flight  path  recordings. 

A-2.2  Visual  System 

A  visual  system  capable  of  processing  and  displaying  a  highly  detailed  tactical 
database  including  multi  level  texture  is  essential  for  a  force  cueing  evaluation.  The 
visual  display  must  have  a  wide  field-of-view  in  order  to  provide  vection  cues  to  the 
pilot.  The  visual  system  must  provide  the  fidelity  and  cues  required  for  low  altitude 
flight,  judging  closure,  and  other  visual  tasks  in  order  that  the  effects  of  force  cueing  may 
effectively  be  determined  without  being  effected  by  inadequate  visual  cueing.  A  visual 
database  must  be  available  which  provides  typical  topography  for  low  altitude  missions 
over  rough  terrain. 

A-3.0  Evaluation  Missions/Maneuvers 

Since  the  ultimate  goal  of  a  force  cueing  evaluation  is  to  determine  the  usefulness 
of  force  cueing  for  tactical  combat  training  simulation,  tactical  missions  must  be  used  for 
an  evaluation.  Guidance  with  respect  to  potential  mission  tasks  and  maneuvers  is 
provided  below. 

A-3.1  Types  of  Missions/Tasks 

1)  Air-to-air  combat. 

2)  Air-to-ground  combat. 

3)  Aerial  refueling/close  formation. 

4)  Landing  and  takeoff. 

5)  Pylon  course. 

6)  Variation. 

-Day  versus  night. 

-Low  versus  medium  altitude. 

A-3.2  Potential  Force  Sensitive  Maneuvers 

Although  an  evaluation  is  to  be  conducted  in  a  mission  context,  it  is  expected  that 
the  pilot’s  performance  of  certain  tasks  and  maneuvers  will  be  more  sensitive  to  force 
cueing  inputs  than  other  tasks.  A  list  of  potential  maneuvers  is  provided  below: 

1)  Closure  on  other  vehicle. 

2)  Maintain  separation  (position)  on  other  vehicle. 

3)  Maintain  low  altitude. 

4)  Perform  ridge  crossing. 
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5)  Avoid  threat. 

6)  Pull  up  into  pop-up. 

7)  Track  airborne  target. 

8)  Track  ground  target. 

9)  Pull  out  after  ground  attack. 

10)  Pitch/roll  captures. 

A-3.2.1  High  Gain  Tracking 

A  force  evaluation  should  include  looking  at  the  effect  of  different  force  cues  on 
performance  of  high  gain  tracking  tasks.  Several  are  provided  below  in  the  order  of  their 
importance.  These  tasks  may  be  performed  with  and  without  turbulence. 

A-3.2.1.1  Close  Formation,  refueling,  etc. 

Performing  tasks  such  as  establishing  and  maintaining  position  during  close 
formation  and  possibly  aerial  refueling  is  one  of  the  highest  gain  tasks.  It  may  be 
affected  by  small  subtle  force  cues  such  as  surge  and  heave. 

A-3.2.1.2  Air  to  air  combat  weapons  delivery. 

Tracking  of  another  aircraft  leading  to  air-to-air  combat  weapons  delivery  is  one 
of  the  higher  gain  tactical  combat  tasks.  This  task  should  be  performed  with  various 
levels  of  disturbance  cues. 

A-3.2.1.3  Air  to  Ground  Weapons  Delivery. 

Tracking  of  a  ground  target  leading  to  delivery  of  a  weapon  is  also  a  high-gain 
tracking  task  similar  to  air  to  air  target  tracking. 


A-3.2.1.4  ILS 


An  ILS  approach  and  landing  is  a  relatively  high  gain  task.  In  addition  to 
turbulence,  this  may  be  performed  with  a  crosswind. 

A-3.2.2  Maintain/Change  Flight  Path 

The  tasks  associated  with  maintaining  and  changing  the  flight  path  of  an  aircraft 
are  generally  lower  gain  than  the  tasks  discussed  under  3.2.1.  However,  the  pilot,  may  be 
significantly  sensitive  to  force  cues  during  performance  of  such  maneuvers. 
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A-3.2.2.1  Low  Altitude  Flight 


A  low  altitude  mission  will  be  flown  with  different  force  cue  configurations.  This 
may  include  both  terrain  following  and  terrain  avoidance.  Performance  factors  may 
include  altitude  maintenance,  ability  to  perform  ridge  crossing,  etc.  Behavior  and 
performance  should  be  compared  for  different  force  cue  inputs. 

A-3.2.2.2  Slalom  Course 

Although  a  slalom  course  may  not  be  representative  of  the  real  world,  it  was 
shown  during  the  Trial  Force-Eval  to  provide  an  effective  tool  for  evaluating  a  force 
cueing  system.  Pilots  should  be  required  to  fly  through  an  airborne  slalom  course  made 
up  of  pylons.  Force  cueing  conditions  can  be  varied  and  pilot  performance  and  may  be 
compared  for  the  different  force  cue  inputs.  Although  the  slalom  course  is  not  part  of  a 
tactical  mission,  it  may  provide  insight  as  to  the  effect  of  various  force  cues  on  pilot 
performance  during  this  general  type  of  aircraft  maneuvering. 

A-4.0  Second  Cockpit 

A  second  cockpit  has  been  shown  to  be  an  essential  part  of  Vis-Eval.  It  provides 
for  a  wingman  for  the  basic  two-ship  element  and  thus  the  ability  to  fly  a  basis  two-ship 
mission.  The  Vis-Eval  results  with  a  second  cockpit  are  more  representative  of  real 
world  flight.  It  is  recommended  that  a  second  cockpit  be  used  in  any  future  force  cue 
evaluations. 
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APPENDIX  B 


RECOMMENDED  PROCESS 
FOR  CONDUCTING  A  FORCE-EVAL 

The  Force-Eval  method  must  be  designed  to  collect  both  subjective  (pilot  opinion) 
and  objective  (measured  performance)  data  for  determining  the  effects  of  force  cues  and 
their  fidelity  upon  pilot  control  strategies,  performance  and  training.  Figure  B-l  provides 
a  block  diagram  of  the  method  or  process  for  conducting  a  force  cueing  evaluation. 
Starting  on  the  left,  based  upon  the  mission,  tasks  to  be  accomplished  are  defined.  Tasks 
which  are  believed  to  be  force  cue  sensitive  are  highlighted  for  more  detailed  tracking 
during  the  force  evaluation.  Flight  profiles  for  these  tasks  may  also  be  delineated.  Both 
the  collected  objective  data  and  expert  pilot  opinion  need  to  be  contributors  to  evaluation. 
Initially,  the  feedback  loops  shown  in  would  be  limited  to  subjective  inputs.  However, 
during  the  evaluation  both  subjective  and  objective  feedback  should  play  a  role  in 
refinement  and  updating  the  evaluation  method.  Again  referring  to  Figure  B-l ,  there  is  a 
series  of  elements  (in  addition  to  the  mission  task/profile  database)  which  is  first 
established  subjectively  using  expert  pilot  and  technical  inputs.  These  elements  include 
the  questionnaires,  pilot  debrief  process,  objective  data  requirements  and  instrumentation. 
The  requirements  for  each  of  these  elements  is  part  of  the  method  which  may  be  further 
defined,  refined  and  validated  during  an  evaluation.  Details  of  these  elements  are 
provided  in  the  sections  that  follow. 

B-l  Force  Cue  Mission  Tasks  Data  Base 

This  database,  when  fully  developed,  identifies  the  force  cues  associated  with 
mission  tasks  including  classification  of  the  type  of  force  cue  (i.e.  maneuver,  disturbance, 
etc.)  and  subjective  ratings  of  its  importance.  Table  B-l  is  an  initial  database,  which 
includes  potentially  force  cue  sensitive  tasks.  It  includes  tactical  combat  tasks,  the  force 
cues  encountered,  the  effects  of  the  force  cues,  the  frequency  with  which  they  are 
experienced  and  non-force  cue  sources  which  may  be  present.  This  table  does  not  include 
cue  classification.  Classification  will  require  evaluation  data.  Table  B-2  provides  a 
strawman  for  task/force  cue  classification.  The  chart  categorizes  the  cues  by  both  type 
and  importance. 

The  mission  task  database  when  more  completely  developed  would  also  describe 
the  flight  profiles  and  how  force  cues  may  be  experienced  during  the  different  task 
maneuvers.  A  force  evaluation  provides  an  opportunity  to  observe,  in  a  real-time  mission 
simulation  environment,  pilot  reaction  to  these  forces.  During  the  evaluation  there  should 
be  an  opportunity  to  expand,  update  and  refine  the  mission  task  data  base  which  may 
have  been  subjectively  compiled  at  the  beginning  of  the  evaluation. 
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B-2  Pilot  Evaluation  Questionnaire 

Referring  to  Figure  B-l,  the  pilot  evaluation  questionnaire  is  developed  based 
upon  the  mission  task/profile  database.  The  effectiveness  of  this  questionnaire  can  only 
be  determined  through  its  application  in  an  evaluation.  Immediately  following  an 
evaluation  mission,  the  pilot  should  complete  the  questionnaire.  The  questionnaire  must 
also  be  updated  as  a  result  of  any  changes  to  the  mission  task/profile  database. 

B-3  Pilot  Debrief  Process 

The  pilot  debrief  is  used  to  expand  upon  and  detail  questionnaire  responses.  The 
debriefing  process  should  be  updated  during  the  Trial  Force-Eval  based  upon  experience 
gained  through  debriefing  the  operational  pilots. 

B-3.1  Follow-up  Pilot  Interview 

Following  review  of  the  recorded  mission  data,  the  training  systems  analysts  may 
choose  to  recall  the  evaluation  pilot  to  review  certain  aspects  of  the  mission.  During  this 
interview,  the  objective  data  collected  may  be  discussed  with  the  pilot  in  order  to  sort  out 
certain  aspects  of  the  results 

B-4  Evaluation  Data  Analysis 

The  evaluation  data  analysis  is  the  process  that  includes  compiling  the  subjective 
data  from  pilot  questionnaires  and  pilot  debriefing  with  objective  data  measured  during 
pilot  evaluation  flights  and  analyzing  the  results.  This  analysis  will  not  have  full 
meaning  until  such  time  as  the  initial  results  are  shared  with  the  evaluation  pilots  during 
the  evaluation  consensus  meeting.  Referring  to  Figure  B-l,  the  Pilot  Operational 
Performance  Data  block,  refers  to  any  data  available  from  aircraft  flight  test  that  may 
have  a  bearing  on  the  analysis.  Since  it  is  not  known  to  what  extent,  if  any,  such  data 
may  be  available,  this  block  is  connected  with  a  dotted  line. 

B-5  Evaluation  Consensus  Meetings 

The  purpose  of  the  evaluation  consensus  meetings  is  to  reach  an  agreement  among 
an  evaluation  team  made  up  of  training/system  engineers  and  expert/operational  pilots,  to 
verify  both  the  evaluation  method  and  the  results  of  the  evaluation.  It  is  conceivable  that 
there  may  be  more  than  one  evaluation  consensus  meeting.  Many  components  of  the 
evaluation  process  may  change  as  a  result  of  information  learned  during  these  consensus 
meetings.  This  may  include  changes  to  the  mission  task/profiles  database,  objective  data 
base  measurement  requirements,  instrumentation,  pilot  debrief  process,  and  evaluation 
data  analysis  process. 
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B-6  Results  and  Validated  Force-Evaluation 


The  results  of  the  Evaluation  consensus  meeting  and  the  analysis  conducted  prior 
to  that  meeting  form  the  basis  of  the  force  evaluation  results.  Referring  to  Figure  B-l, 
there  are  several  feedback  loops,  which  during  the  course  of  the  evaluation  should  assist 
in  fine  tuning  the  results. 

B-7  Force  Evaluation  Data  Organization 

Conducting  a  force  evaluation  will  result  in  the  collecting  of  large  amounts  of 
both  objective  and  subjective  data.  Organization  of  the  data  for  analysis  is  extremely 
critical  to  timely  data  reduction.  Also,  an  effective  data  management  system  is  required 
for  facilitating  future  use  of  the  method. 
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Figure  B-l.  Force  Cueing  Evaluation  Method 


_ _  Table  B-l.  Force  Cue  Mission  Tasks  Data  Base 
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Table  B-l 

Force  Cue  Mission  Tasks  Data  Base 
(continued) 
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Table  B-l 

Force  Cue  Mission  Tasks  Data  Base 
(continued) 
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aircraft  in  his  flight—an  even  more 
difficult  task. 


Table  B-2  FORCE  CUE  IMPORTANCE  RATINGS 
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GLOSSARY 


Cue.  A  stimulus  that  has  acquired  meaning  (Caro,  1988). 

Cutaneous.  Pertaining  to  the  skin  or  receptors  in  the  skin,  or  to  sensation  mediated  by 
receptors  in  the  skin  (Boff,  1988). 

Discrimination.  The  recognition  that  a  given  stimuli  or  response  has  a  meaning  different 
from  that  of  another  stimulus  or  response  (Caro,  1988). 

Dynamic  Seat.  A  simulated  aircraft  seat  designed  to  use  in  a  flight  simulator  which 
provides  the  pilot  with  a  wide  range  of  force  cues  such  as  sustained  high-g  force  cues, 
vibration,  and  tactual  cues.  It  may  also  include  anti  g-suit  or/and  lap  belt  actuation. 

g-Seat.  A  simulated  aircraft  seat  designed  to  use  in  a  flight  simulator  which  provides  the 
pilot  with  sustained  high-g  force  cues.  Simulated  seats  with  the  additional  capabilities  of 
a  dynamic  seat  are  often  referred  to  as  g-seats. 

Haptic.  Pertaining  to  or  arising  from  tactual  perception  based  upon  cutaneous  and 
kinesthetic  information.  (Boff,  1988). 

Hz.  Hertz,  a  unit  of  frequency  equal  to  one  cycle  per  second. 

Kinesthetic.  The  sense  of  movement  and  position  of  limbs  or  other  body  parts,  arising 
from  stimulation  of  receptors  in  the  joints,  muscles,  and  tendons.  (Boff,  1988). 

Stimuli.  A  physical  object  or  event  that  can  activate  a  sense  organ.  Stimuli  are  the  bases 
for  cues  (Caro,  1988). 

Tactile.  Of  or  relating  to  tactual  perception  (touch)  mediated  by  the  cutaneous  (skin) 
sense.  (Boff,  1988). 

Tactual.  Of  or  relating  to  the  sense  of  touch,  as  mediated  by  the  cutaneous  (skin)  sense 
and/or  kinesthetic  (Martin,  1986). 

Vection.  Vection  is  an  illusory  sensation  of  motion  of  the  self,  produced  by  translatory 
and  angular  motions  of  the  visual  scene.  (Boff,  1988). 

Vestibular  Sense.  The  sense  mediated  by  the  otolith  organs  and  semi-circular  canals  that 
is  concerned  with  the  perception  of  head  position  and  motion  and  is  stimulated  by 
acceleration  associated  with  head  movements  and  changes  in  the  pull  of  gravity  relative 
to  the  head.  (Boff,  1988). 
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Vestibular  System.  The  system  comprised  of  the  otolith  organs  and  the  semi-circular 
canals  that  mediates  the  perception  of  head  position  and  motion.  (Boff,  1988). 
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LIST  OF  ACRONYMS 


A/A 

Air-to-Air 

A/G 

Air-to-Ground 

AOA 

Angle  of  Attack 

ALCOGS 

Advanced  Low  Cost  g-Seat 

BVR 

Beyond  Visual  Range 

Combat  Edge 

Combined  Advanced  Technology  Enhanced  Design  G 
Ensemble 

EEG 

FOV 

Field  of  View 

HDD 

Head  Down  Display 

HOTAS 

Hands  On  Throttle  And  Stick 

HUD 

Head-Up  Display 

Hz 

Hertz 

I/O 

Input/Output 

LCOSS 

Lead  Computed  Optical  Sight  System 

LOC 

Loss  of  Consciousness 

LOS 

Line  of  Sight 

MPD 

Multipurpose  Display 

MRM 

Medium  Range  Missile 

NED 

North-East-Down 

nm  or  NM 

Nautical  Miles 

NZ 

Normal  Acceleration 

OTW 

Out  The  Window 

PDT 

Primary  Designated  Target 

PPBG 

Positive  Pressure  Breathing  System 

PVI 

Pilot  Vehicle  Interface 

SA 

Situation  Awareness 

SD 

Situation  Display 

SRM 

Short  Range  Missile 

USAF 

United  States  Air  Force 

WL 

Wright  Laboratory 

WOW 

Weight  on  Wheels 

WPAFB 

Wright  Patterson  Air  Force  Base 

WVR 

Within  Visual  Range 
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